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TITLE: METHODS FOR PERFORMING DAF DATA FILTERING 

AND PADDING 



BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates to methods using distributed approximating 
functionals (DAF), DAF-wavelets and DAF-neural networks for filtering, denoising, 
5 processing, restoring, enhancing, padding, or other signal processing procedures directed 

to images, signals, ID, 2D, 3D . . . nD spectra, X-ray spectra, CAT scans; MRI scans, 
NMR, and other applications that require data processing at or near the theoretical limit 
of resolutions. 

More particularly, the present invention relates to the use of infinitely smooth 
10 DAFs in combination with other signal processing techniques to provide methods and 

apparatuses utilizing such methods that can enhance image, spectral, or other signal data 
and decrease the time need to acquire images, spectra or other signals, 

2. Description of the Related Art 

15 Many techniques currently exist for processing images, refining spectra, 

analyzing data or the like. Many of these techniques are well-known and used 
extensively. However, these techniques generally suffer from one or more limitation on 
their ability to enhance signal or image and construct or restore missing or lost data, 
especially if the user desires the error inherit in signal acquisition and the error introduced 

20 by the processing technique to be as small as possible, z.e., as close as possible to 

Heisenberg's uncertainty principle. 

Thus, there is a need in the art for improved techniques for processing acquired 
data whether in the form on a image, a spectra, a multidimensional spectra or the like so 
that the error due to processing can be minimized which can increase resolution and 

25 decrease acquisition times. 
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SUMMARY OF THE INVENTION 

The present invention provides a method implemented on a digital processing 
device or stored in a memory readable by a digital processing unit which uses distributed 
approximating functionals (DAFs) to enhance and improve signal, image and multi- 
5 dimensional data constructs processing and to decrease acquisition time of real world 

spectrometric techniques that operate on square wave type frequency domains which 
require a large acquisition time to capture the signal from noise. Short the acquisition 
time, the more noise and less resolution and definition the spectra will have. The limit 
for simultaneously acquiring frequency information in time is given by a variant of 

10 Heisenberg's uncertainty principle, i.e., AcoAt^ 1 . The methods of the present provides 

methods for image and signal processing where the accuracy and precision of the final 
signal and image closely approaches the uncertainty principles maximum accuracy and 
precision. The methods can be made to approach uncertainty principle accuracy via 
increased computational cost, but the real power of the methods of this invention is to 

15 yield improved measurements at a given Aco and At 

product. 

The present invention also provides methods for improving X-ray and magnetic 
imaging techniques, especially mammogram images using the DAP and D AF processing 
techniques set forth herein. 
20 The present invention also provides a mammogram imaging system of 

conventional design, the X-ray data derived thereform is then enhanced via DAF 
processing in an associated digital processing unit. 
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DESCRIPTION OF THE DRAWINGS 

The invention can be better understood with reference to the following detailed 
description together with the appended illustrative drawings in which like-elements are 
numbered the same. The figure are heading dependent. Each major heading set forth by 
a all caps, embolded and underlined text has its own figures each restarting each 
numbering sequence with 1. Each major heading also restarts equation numbering. 
D AF TREATMENT OF NOISY SIGNALS 

Figure 1 depicts the Hermite DAF in (a) coordinate space and (b) frequency space 
respectively. The solid line is for $M=8 8,-\sigma=3 .05$ and the dashed line is for A/=l 2, 
-0=4. The solid line is close to the interpolation region and the dashed Vine is in the low 
pass filtering region. The frequency in (b)has been multiplied by a factor of the grid 
spacing. 

Figure 2 depicts extrapolation results for the ftinction in Eq. (15). The solid line is the 
exact function. The long dashed Une, the short dashed line and the plus symbols are the 
extrapolation results for Hermite DAF parameters a/A=7.6, -7.8, and 8.0 respectively. 
In our numerical application, only the values at the grid points less than -1 .2 are assumed 
to be known. 

Figure 3 depicts sine function in Eq. (17) with 50% random noise added to the values at 
even spaced grids from 0 to 219 (solid line) and the periodically extended 36 function 
values (plus symbols) with o/A=10.5. The exact values in the extended domain are also 
plotted (solid line) in this figure. 

Figure 4 depicts the error and (b) the signature of the periodic padding of the noisy 

sine function as a function of DAF parameter a/A. The Mis fixed to be-6. 

Figure 5(a) depicts the error and Figure 5(b) the signature of the DAF smoothing to 

the periodically extended noisy sine function as a function of o/A. The M is fixed to be 

12. 

Figure 6 depicts periodic extension of the nonperiodic function (with noised added) in 
Eq. ( 1 5). (a) The 220 known values of the function in the range [-7, 1 0] with 20% random 
noise (solid line) and the 36 exact values of the function(dashed line). Note that the 
function is not periodic at all in the range of 256 grid points, (b) The- periodically 
extended function with o/A=5.2. Note the smoothness and periodic property of the 
function. 
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Figure 7 depicts the periodic extension signature of the noisy function in Figure 5as a 
function of o/A. The M is fixed to be 6. 

Figure 8(a) depicts the L„ error and Figure 8(b) the signature of the DAF SMOOTHING 
to the periodically extended noisy function in Figure 6(b) as a function of o/A. The Mis 
fixed to be 12for the DAF-smoothing. 

Figure 9 depicts a comparison of DAF smoothed result to the signal in Figure 6(b) at 

o/A=9 (solid line), and with the exact function (dashed line) in Eq. (15). 

GENERALIZED SYMMETRIC INTERPOLATING WAVELETS 

Figure 1 . -/rband-limited interpolating wavelets (a) Sine function and (b) Sinclet wavelet; 

Figure2. Interpolating Cardinal Spline (I>=5); . " - 

Figures. Interpolating wavelets by auto-correlation shell (i>=3) (a) Daubechies wavelet 

(b) Dubuc wavelet; 

Figure 4. Lifting scheme 

Figures. Lagrange Wavelets with Z)=3 (a) ScaUng function (b) Wavelet (c) Dual scaling 
function (d) Dual wavelet; 

Figure6. Frequency Response of Equivalent Filters (Z)=3) (a) Decomposition (b) 
Reconstruction; 

Figure?. Non-regularized Lagrange Wavelets (A/=5) (a) Lagrange polynomial (b) 
Scaling function (c) Wavelet (d) Dual scaling function (e) Dual wavelet 
Figures. B-Spline Lagrange DAF Wavelets (AM, h=2) (a) Scaling function (b) 
Wavelet (c) Dual scaling function (d) Dual wavelet 

Figure9. Frequency Response ofEquivalent Filters (AM, h=2) (a) Decomposition (b) 
Reconstruction 

Figure! 0. Mother wavelet comparison (AM, h=2) Solid: B-spline Lagrange; dotted: 
Gaussian-Lagrange 

Figure 1 1. Gibbs overshoot of the Sine FIR 

Figurel2. Sine Cutoff Wavelets (M=9) (a) Scaling (b) Wavelet (c) Dual scaling (d) 
Dual wavelet 

FigurelS. B-Spline Lagrange DAF Wavelets (A^=5, h=3) (a) Scaling (b) Wavelet (c) 
Dual scaling (d) Dual wavelet 

Figurel4. Frequency Response of Equivalent Filters (A^=5, h=3) (a) Decomposition (b) 
Reconstruction 
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FigurelS. Mother Wavelet Comparison (A^=4, h=2) Solid: B-spline Sine; dotted: 
Gaussian Sine 

Figurel6 Dubuc wavelets {D=3) (a) Inner scaling (b) Boundary scaling . * 
Figurel 7 Generalized Sine wavelets (^=2, h=2) (a) Inner scaling (b) Boundary scaling 
5 FigurelS Frequency response comparison of boundary filters (a) Halfband Lagrange 

wavelet (b) Sinc-DAF wavelet 

Figurel9 Target extraction from color background (a) Original pilot view (b) DAF- 
wavelet restoration 

Figure20. Visual Color Image Restoration (a) Noisy girl (b) Our restoration 
10 Figure21. Enhancement of Database 1 (a) Original Mammogram ' ^- '(b) Linear 

enhancement (c) Non-linear enhancement 

Figure22. Enhancement of Database 2 (a) Original mammogram (b) Enhancement 
DUAL WINDOW SELECTIVE AVERAGING FILTER 

Figure 1 depicts the filtering of blocked signal corrupted by Gaussian noise: (a) 
1 5 noise-free blocks signal; (b) noisy signal (MSE=1 .00, MAE=0.80); and (c) filtered signal 

by DWSAF (MSE=6.19E-2, MAE=6.49E-3). 

Figure 2 depicts image restoration from lower noise Lena image: (a) corrupted image 
(PSNR=22.17 dB) and (b) restored image by DWSAF (PSNR=30.69 dB). 
Figure 3 depicts image restoration from higher noise Lena image; (a) corrupted image 
20 (PSNR=18.82 dB) and (b) restored image by DWSAF (PSNR=28.96 dB), 

LAGRANGE WAVELETS FOR SIGNAL PROCESSING 
Figure 1 depicts 7t. band-limited interpolating wavelets (a) Sine function (b) Sinclet 
wavelet; 

Figure 2 depicts Interpolating Cardinal Spline (i>=5); 
25 Figure 3 depicts Interpolating wavelets by auto-correlation shell (i>=3) (a) Daubechies 

wavelet (b) Dubuc wavelet 
Figure 4 depicts Lifting scheme 

Figure 5 depicts Lagrange Wavelets with D=3 (a) Scaling (b) Wavelet (c) Dual scaling 
(d) Dual wavelet 

30 Figure 6 depicts Frequency Response of Equivalent Filters (D=3) (a) Decomposition (b) 

Reconstruction 

Figure 7 depicts Lagrange Wavelets with D=9 (a) Scaling (b) Wavelet (c) Dual sealing - 
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(d) Dual wavelet 

Figure 8 depicts Frequency Response of Equivalent Filters (D=9) (a) Decomposition (b) 
Reconstruction . - 

Figure 9 depicts Non-regularized Lagrange Wavelets (M=5) (a) Lagrange polynomial 
5 (b) Scaling (c) Wavelet (d) Dual scaling (e) Dual wavelet 

Figure 10 depicts B-Spline Lagrange DAF Wavelets (N=4, h=2) (a) Scaling (b) 
Wavelet (c) Dual scaling (d) Dual wavelet 

Figure 11 depicts Frequency Response of Equivalent Filters (Af=4, h=2) (a) 

Decomposition (b) Reconstruction 
10 Figure 12 depicts Mother Wavelet Comparison (N~4, h=2) Solid: B-splme Lagrange; 

dotted: Gaussian Lagrange * - . • 

Figure 13 depicts Nonlinear Masking Fxmctionals (a) Donoho Hard Logic Nonlinearity 
(b) Softer Logic Nonlinearity 

Figure 14 depicts 2D Lagrange wavelets for image processing (a) Scaling, (b) Vertical, 
15 (c) Horizontal and (d) Diagonal wavelets 

Figure 15 depicts VGN image processing for Lena (a) Noisy Lena (b) Median filtering 

result (c) our method 

Figure 16 depicts VGN processing for Barbar (a) Noisy Barbara (b) Median filtering 
result (c) Our method 

20 IMAGE ENHANCEMENT NORMALIZATION 

Figure 1(a) Original Mammogram and Figure 2(b) Enhanced result 

Figure 2(a) Original Mammogram and Figure 2(b) Enhanced result ' 

VARYING WEIGHT TRIMMED MEAN FILTER FOR THE RESTORATION OF 

IMPULSE CORRUPTED IMAGES 

25 Figure 1 depicts the weight function (5) for A=2 

Figure 2 depicts image restoration firom 40% impulse noise corrupted Lena image. Figure 
2(a) shows the original Lena picture; Figure 2(b) shows noise image; Figure 2(c) shows 
median Filtering (3X3), PSNR=28.75; Figure 2(d) a-TMF (3X3), PSNR=27.49; Figure 
2(e) VWTMF (3X3), PSNR=29.06; and Figure 2(f) VWTMF - switch (3X3), 

30 PSNR=31.43. 

A NEW NONLINEAR IMAGE FILTERING TECHNIQUE 

Figure 1 Image restoration firom 60% impulse noise, (a) Corrupted image, (b) Filtering' • 
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by Sun and Nevou's median switch scheme, (c) Our filtering, (d) Our modified filtering. 
Figure 2 Image restoration from 40% impulse noise, (a) Corrupted image, (b) Median 
filtering (3 '3).(c) Median filtering (5 '5). (d) Our filtering. 

BIOMEDICAL SIGNAL PROCESSING USING A NEW CLASS OF WAVELETS 
5 Figure 1 

Figure 2a 
Figure 2b 
Figure 3 
Figure 4(a) 
10 Figure 4(b) 

Figure 4(c) 

VISUAL MULTIRESOLUTION COLOR IMAGE RESTORATION 

Figure 1 
Figure 2 
15 Figures 

NONLINEAR QUINCUNX FILTERS 
Figure 1 
Figure 2 
Figures 

20 Figures 4, 4(a), 4(b) 

Figures 5, 5(a), 5(b) 
Figures 6(a-c) 

MAMMOGRAM ENHANCEMENT USING GENERALIZED SINC WAVELETS 
Figure 1. % band-limited interpolating wavelets 
25 (a) Sine function (b) Sinclet wavelet 

Figure2, Fourier Gibbs overshot of Sine FIR implementation 

Figure 3. Sine Cutoff Wavelets (Af=9) (a) Scaling (b) Wavelet (c) Dual scaling (d) 
Dual wavelet 

Figure4. B-Spline Lagrange DAF Wavelets (//=5, h=3) (a) Scaling (b) Wavelet (c) 
30 Dual scaling (d) Dual wavelet 

Figure5. Frequency Response of Equivalent Filters (A^=5, h=3) (a) Decomposition (b) 
Reconstruction 
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Figure 6. Mother Wavelet Comparison (A/M, h=2) Solid: B-spline Sine; dotted: 
Gaussian Sine 

Figure 7. Nonlinear Masking Functionals (a) Donoho Hard Logic Nonlinearity (b) 
Softer Logic Nonlinearity 

Figure 8. Mammogram enhancement (a) Original mammogram (b) Multiresolution 
enhancement by DAF-wavelet 

VISUAL MULTIRESOLUTION COLOR IMAGE RESTORATION 

Figure L Cube model of RGB color 
Figure2. Alternative representation 
Figures. Hexagon projection of color tube 

Figure4 Tested Result (1) (a) Noisy Lena (b) Median filtering (c)VGN restoration 
Figures. Tested Result (2) (a) Noisy girl (b) Median filtering (c) VGN restoration 
DUAL PROPAGATION INVERSION OF FOURIER AND LAPLACE SIGNALS 
Fig. 1 The auxiliary function, (t;a,a)_0), at t=0, as a function of the frequency O)_0. 
Fig. 2 The truncated sine function f(G))=sin(a>); O^o)^\pi, and the calculated spectrum 
obtained by the dual propagation inversion procedure. The noiseless time domain signal 
was sampled between -45^t^45. The two are visually indistinguishable. 
Fig. 3 Dotted line: The calculated spectrum f(a)) obtained from the time signal corrupted 
by random noise of 20%. Solid line: The calculated spectrum obtained firom the 
noise- free time signal. Both clean and corrupted signals were sampled between -45^t^45, 
Fig. 4 Cross-hatched line: the calculated spectrum /dpi(co) obtained from ttie noise-free 
time domain signal, sampled between -5^ts5. Solid line is the original truncated sine 
function. 

Fig. 5 Cross-hatched line: the calculated spectrum /dpiCw) obtained using the 
DAF-padding values for 5^|t|^7.5, joined smoothly to the analytical tail-function (see 
text). Careful comparison with Fig. 4 shows a reduction of the aliasing due to signal 
truncation. 
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DETAILED DESCRIPTION OF THE INVENTION 

The inventors have found that a signals, images and multidimensional imaging 
data can be processed at or near the uncertainty principle lunits with DAFs and various 
adaptation thereof which are described in the various section of this disclosure. 

D AF TREATMENT OF NOISY SIGNALS 
INTRODUCTION 

Experimental data sets encountered in science and engineering contain noise due 
to the influence of internal interferences and/or external environmental conditions. 
Sometimes the noise must be identified and removed in order to seethe true signal 
clearly, to analyze it, or to make further use of it. , . 

Signal processing techniques are now widely applied not only in various fields 
of engineering but also in physics, chemistry, biology, and medicine. Example problems 
of interest include filter diagonalization, solvers for eigenvalues and eigenstates [1-3], 
solution of ordinary and partial differential equations [4-5], pattern analysis, 
characterization, and denoising [6-7], and potential energy surface fitting [8], One of the 
most important topics in signal processing is filter design. Frequency-selective filters are 
a particularly important class of linear, time-invariant (LTI) analyzers [9]. For a given 
experimental data set, however, some fi-equency selective finite impulse response (FIR) 
filters require a knowledge of the signal in both the unknown "past" and "fixture" 
domains. This is a tremendously challenging situation when one attempts to analyze the 
true signal values near the boundaries of the known data set. Direct application of this 
kind of filter to the signal leads to aliasing; z.e., the introduction" of additional, 
nonphysical firequencies to the true signal, a problem called "aUasing" [9]. Additionally, 
in the implementation of the fast Fourier transform (FFT) algorithm [9], it is desirable 
to have the number of data values or samples to be a power of 2. However, this condition 
is often not satisfied for a given set of experimental measurements, so one must either 
delete data points or augment the data by simulating in some fashion, the unknown data. 

Determining the true signal by extending the domain of experimental data is 
extremely difficult without additional information concerning the signal, such as a 
knowledge of the boundary conditions. It is an even tougher task, using the typical 
interpolation approach, when the signal contains noise. Such interpolation formulae 
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necessarily return data values that are exact on the grids; but they suffer a loss of 
accuracy off the grid points or in the unknown domain, since they reproduce the noisy 
signal data exactly, without discriminating between true signal and the. noise. In this 
paper, an algorithm is introduced that makes use of the well-tempefed property of 
"distributed approximating functionals" (DAFs) [10-13]. The basic idea is to introduce 
a pseudo-signal by adding gaps at the ends of the known data, and assuming the 
augmented signal to be periodic. The unknown gap data are determined by solving linear 
algebraic equations that extremize a cost function. This procedure thus imposes a 
periodic boundary condition on the extended signal. Once periodic boundary conditions 
are enforced, the pseudo-signal is known everywhere and can be usedTdr a variety of 
numerical applications. The detailed values in the gap are usually not of particular 
interest. The advantage of the algorithm is that the extended signal adds virtually no 
aliasing to the true signal, which is an important problem in signal processing. Two of 
the main sources of aliasing are too small a sampling frequency and truncation of the 
signal duration. Another source of error is contamination of the true signal by numerical 
or experimental noise. Here we are concerned only with how to avoid the truncation 
induced and noisy aliasing of the true signal. 

Distributed approximating functions (DAFs) were recently introduced [ 1 0- 1 1 ] as 
a means of approximating continuous functions from values known only on a discrete 
sample of points, and of obtaining approximate linear transformations of the function (for 
example, its derivatives to various orders). One interesting feature of a class of 
commonly-used DAFs is the so-called well-tempered property [13]; it is the key to the 
use of DAFs as the basis of a periodic extension algorithm. DAFs differ from the most 
commonly used approaches in that there are no special points in the DAF approximation; 
z. e. , the DAF representation of a function yields approximately the same level of accuracy 
for the function both on and off the grid points. However, we remark that the 
approximation to the derivatives is not, in general quite as accurate as the DAF 
approximation to the function itself because the derivatives of Z^- functions contain an 
increased contribution from high frequencies. By contrast; most other approaches yield 
exact results for the function on the grid points, but often at the expense of the quality of 
the results for the function elsewhere [13]. DAFs also provide a well-tempered 
approximation to various linear transformations of the function. DAF representations of- 
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derivatives to various orders will yield approximately similar orders of accuracy as long 
as the resulting derivatives remain in the DAF class. The DAF approximation to a 
function and a finite number of derivatives can be made to be of machine accuracy with 
a proper choice of the DAF parameters. DAFs are easy to apply because they yield 
5 integral operators for derivatives. These important features of DAFs have made them 

successful computational tools for solving various linear and nonlinear partial differential 
equations (PDEs) [14-17], for pattern recognition and analysis [6], and for potential 
energy surface fitting [8]. The well-tempered DAFs also are low-pass filters. In this 
paper, the usefiilness of DAFs as low pass filters is also studied when they are applied 

10 to a periodically extended noisy signal. For the present purpose, we assumelhat the weak 

noise is mostly in the high fi-equency region and the true signal is bandwidth limited in 
firequency space, and is larger than the noise in this same frequency region. To determine 
when the noise is eliminated, we introduce a signature to identify the optimum DAF 
parameters. This concept is based on computing the root-mean-square of the smoothed 

15 data for given DAF parameters. By examining its behavior as a function of the DAF 

parameters, it is possible to obtain the overall frequency distribution of the original noisy 
signal. This signature helps us to periodically extend and filter noise in our test examples. 

The first example is a simple, noisy periodic signal, for which the DAF periodic 
extension is a special case of extrapolation. The second is a nonperiodic noisy signal. 

20 After performing the periodic extension and filtering, it is seen that the resulting signal 

is closely recreates the true signal. 

DISTRIBUTED APPROXIMATING FUNCTIONALS 

DAFs can be viewed as "approximate identity kernels" used to approximate a 
25 continuous function in terms of discrete sampling on a grid [10-13]. One class of DAFs 

that has been particularly useful is known as the well-tempered DAFs, which provide an 
approximation to a function having the same order of accuracy both on and off the grid 
points. A particularly useful member of this class of DAFs is constmcted using Hermite 
polynomials, and prior to discretization, is given by 



30 



K f 'I A 1 / (x^xO^ \y?/ ly 1 „ ( {x ^ x') \ (1) 
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where o, Af are the DAP parameters and Hj^ is the usual (even) Hermite polynomial. The 
Hermite DAF is dominated by its Gaussian envelope, exp(-(x-;c')V2 o^), which effectively 
determines the extent of the function. The continuous, analytic approximation to a 
functiony(x) generated by the Hermite DAF is 

5 oo (2) 

f{x) ^ foAFix) = r Sm{x ~ x'\a)f{x')dx\ 

J — oo 

Given a discrete set of functional values on a grid, the DAF approximation to the 
function at any point x (on or off the grid) can be obtained by 

fDAF{^) = A E Sm{x - «y|cr)/(x,). (3) 

i 

where A is the uniform grid spacing (non-uniform and even random sampling can also 
be used by an appropriate extension of the theory). The summation is over all grid points 
(but only those close to x effectively contribute). Similarly, for a two-dimensional 
1 5 functiony(^3(y), one can write 

foAFixyy) = A, Aj, ^ SmAx - xj, \(r,)SM,{y - yj,W2)f{xj, ,yj,), W 

using a simple direct product. In Figure 1, we plot Hermite DAFs obtained with two 
different sets of parameters, in (a) coordinate space, and (b) frequency space. The solid 
line (a=3.05, M=88) is more interpolative compared to the DAF given by the dashed line 
(o=4, A/=12), The latter is more smoothing when applied to those "fuhctions whose 
Fourier composition lies mostly under the o = 3.05, M=88 DAF window. This results 
from the a =4, M=12 DAF window being narrower in Fourier space than that of the DAF 
with 0=3.05, M=88. The discretized Hemiite DAF is highly banded in coordinate space 
due to the presence of the Gaussian envelope, which means that only a relatively small 
number of values are needed on both sides of the point x in Eq. (3), as can be clearly seen 

from Figure 1 (a). This is in contrast to the sine function . ^^"^^^^ . From Figure 1 (b), 

JTX 

we see that the Hermite DAF is also effectively bandwidth-limited in frequency space. 
With a proper choice of parameters, the Hermite DAF can produce an arbitrarily good 
filter (see the dashed line in Figure 1 ). Once the boundary condition is fixed for a data set, ' 
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Eq.(3) or (4) can then be used to eliminate the high frequency noise of that data set. As 
long as the frequency distribution of the noise lies outside the DAF plateau (Figure 1 (b)), 
the Hermite DAF will eliminate the noise regardless of its magnitude. 

The approximate linear transformations of a continuous function can also be 
generated using the Hermite DAF. One particular example is derivatives of a fiunction to 
various orders, given by 

f^'\^) « ISaf = r *S(x - x'\a)f{x')dx', (5) 

J —OO 

where Sl!f\x- x'\ a) is the /th derivative of S^{x- x' \a), with respect to jc, and 
is given explicitly by 




When imiformly discretized by quadrature, Eq. (5) gives 

3 

Expressions (5) and (7) are extremely useful in solving linear and nonlinear partial 
differential equations (PDEs) [14-17] because the differential operation has been 
expressed as an integration. With a judicious choice of the DAF parameters, it is possible 
to provide arbitrarily high accuracy for estimating the derivatives. 

METHOD OF DATA EXTRAPOLATION 
Case I. Filling a Gap 

Suppose we have a set of uniformly spaced grid points on the infinite line, and 
a continuous function, y(x), known on all grid points except for the set {jcj, x^. 
Assuming that f{x) is in the DAF-class, we can estimate the unknown- values by 
minimizing the cost function. 



c= E Wp(/(x,)-w(«p))'. 



(8) 
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where, is a weight assigned to the point x^, and in this paper it is chosen to be 1 on a 
finite grid and 0 eisewhere^^X^p is the DAF approximation to the function at the point 
Xp, From Eqs. (3) and (8), we have 



5 C= f; wJfix^)- E 6Mix,-x,W)fiz,)\ , (9) 

where w is the half DAF bandwidth. We minimize this cost function with respect to the 
unknown values, {f[xj), ...,/Xk)}, according to 

dc 

10 57(^0=°' -^^r^^. (10) 

to generate the set of linear algebraic equations, 

oo / p+w \ 

E 2Wp fix,) - Sm{x, - !Ce|cr)/(xO [S^ - 6m{x, - x,|cr)) = 0, J<l<K, ^^^^ 

p=-oo \ t=p-u) / 

where the unknowns are^x^) andj{x,) for p=l or M. The symbol 4, is the kronecker 
delta. Solving these equations yields the predicted values affix) on the grid points in the 
gap. 

20 Case 11. Extrapolation 

A more challenging situation occurs when/jC;), />Jare all xmknowri. In this case, 
for points x^ beyond x^y we specify a functional form for the unknown-jgrid values, 
including some embedded variational parameters. It is simplest to* choose linear 
variational parameters, e.g., 

25 

L 

/(^) « E <^^M^h (12) 

but this is not essential, and nonlinear parameters can also be embedded in the ^^(x). The 
choice of functions, ^^(x), can be guided by any intuition or information about the 
physical behavior of the signal, but even this is not necessary. This introduces additional 
variations of the cost function with respect to the additional parameters, so we impose 

— =0, 1 ^ Z, and therefore obtain additional equations. We must also specify the 
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choice of the when one introduces both a gap and a ' 'tail function". There is enormous 
freedom in how this is done, and e.g., one can choose which points are included and 
which are not. In the present study, we shall take W^=^l for 1 ^p^K (/.e., all known data 
points, plus all gap-points), and^^n l for all other points(including tail-fimction points). 
Again, we emphasize that other choices are possible and are under study. 

For case I, our procedure leads to Eq. (1 1) and for case II, to the equations, 

X (Jp, - M*p - */k)) = 0, J<i<K, viJJ 
E -zw^p /(xp)- M»=P-x.k)/(x.) 

X ^M{xp - xt)<f>^{xt) = 0, 1<P<L 

These linear algebraic equations can be solved by any of a variety of standard algorithms 
[18]. Note that it is the well-tempered property of the DAFs that underlies the above 
algorithms. For standard interpolation algorithms, the value on each grid point is exact 
and does not depend on the values at other grid points, which means that the cost function 
is always zero irrespective of functional values. 

The suitability of using Hermite DAFs to pad two isolated data sets has been 
tested for fitting one dimensional potential energy surfaces [8]. To explore further the 
algorithm in the case where only one data set is known, we show in Figure 2 the 
extrapolation results for the arbitrarily chosen DAF-class function, 

4 

/W = 4 + 2$3cos(jx) + 5exp(-(x-l)2). (15) 

Using values of the function on a discrete grid with uniform spacing, A ^ 0.024, on the 
domain shown in Figure 2 (solid line),we attempt to determine the function at 100 
uniformly distributed grid points in the range [-1.2, 1.2]. The tail function used isfix)=l, 
multiplied by a linear variational parameter. From Figure 2, it is seen that the predicted 
results are in almost total agreement with the actual function, for all three DAF 
parameters employed, for the points between -1.2^jc^0.2* Larger errors occur for those 
X values which are further away from the known data boimdary. The source-of error 
simply is that one is forcing the function to join smoothly with a constant- tail function, 
even though the constant is variationally determined. Had one employed the correct form - 
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for the tail function, with a multiplicative variational factor, the result would be visually 
indistinguishable for all three DAF parameters and the tail-variational constant would 
turn out to be essentially unity. It must be noted that, although we have discussed the 
algorithm in the context of one dimension, extending it to two. or more dimensions is 
straightforward. One way to do this is with a direct product form, as given in Eq. (4). 
However, such a direct two dimensional calculation is a time and memory consuming 
procedure because of the large number of simultaneous algebraic equations that must be 
solved. One alternative is to consider the two dimensional patterns as a grid constructed 
of many one dimensional grids, and then extrapolate each line separately. We expect this 
procedure may be less accurate than the direct two dimensional extrapofatton algorithm 
because it only considers the influence from one direction and neglects cross correlation. 
However, for many problems it produces satisfactory results and it is a very economical 
procedure. Additionally, cross correlation can be introduced by D AF-fitting the complete 
(known plus predicted) data set using the appropriate 2D DAF. 

The well-tempered property makes the DAFs powerful computational tools for 
extrapolation of noisy data, since DAFs are low-pass filters and therefore remove high 
frequency noise. In the next section, we will explore the use of the algorithm presented 
here for periodically extending a finite, discrete segment of data which may contain 
noise, 

PERIODIC EXTENSION 

As described in the introduction to this paper, sometimes it is necessary to know 
the boundary conditions for a data set in order to apply noncausal, zero-phase FIR filters 
without inducing significant aliasing. Certain other numerical analyses require that the 
signal contain a number of samples equal to an integer power of 2. However, it is most 
often the case that the boundary conditions for the experimental data are unknown and 
the length of the data stream is fixed experimentally and typically not subject to 
adjustment. 

A pseudo-signal is introduced outside the domain of the nonperiodic experimental 
signal in order to force the signal to satisfy periodic boundary conditions and/or-to have 
the appropriate number of samples. The required algorithm is similar to that for filling 
a gap, as discussed above. One can treat the period-length as a discrete variational- 
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parameter but we don't pursue this here. For a given set of experimental data {/i,/2,».,A 
,)}, we shall force it to be periodic, with period K, so that K-J^l values {fjyfj^^.^,,/^} 
must be detemiined. Since the extended signal is periodic, the values {^^-i ,yic+2»"MA+j-i} 



resulting signal can, of course, be infinitely extended as may be required for various 
numerical applications. 

The pseudo-signal is only used to extend the data periodically retaining 
essentially the same frequency distributions. The utility of the present periodic extension 
algorithm is that it provides an artificial boundary condition for the signal without 
significant aliasing. The resulting signal can be used with any filter requiring information 
conceming the future and past behavior of the signal. In this paper, we also employ a 
Hermite DAF to filter out the higher frequency noise of the periodic, extended noisy 
signal. For doing this, we assume that the true signal is bandwidth limited and that the 
noise is mostly concentrated in the high frequency region. 

As shown in the test example extrapolation in section III, there are infinitely 
many ways to smoothly connect two isolated signals using DAFs with different choices 
of the parameters. We require a procedure to determine the optimum DAF parameters in 
a ' 'blind" manner. Fourier analysis is one way to proceed, but due to the structure of the 
Hermite DAF, we prefer to optimize the parameters while working in physical space 
rather than Fourier space. To accomplish this, we introduce a generalized signature for 
both noisy data extension and filtering, which we define to be 



where, M and aare Hermite DAF parameters, and Jdaf is the arithmetic average of the 

foAfi^n) Oh® - oo off^^p(x„)). The signature essentially measures the smoothness of 
the DAF filtered result. 

A typical plot of SJ^dL) is shown in Figure 4(b). We first note that it is 
monotonically decreasing. This is to be expected since increasing results in a 
smoother, more highly averaged signal. The second major feature of interest is the 
occurrence of a broad plateau. In this region most of the noise has been removed from 
the DAF approximation; however, the dominant portion of the true signal is still 



are also, by fiat, known to be equal to {/;,/2,...,/;.,}- Once the ^ap is filled in, the 




(16) 
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concentrated under the DAF frequency window. As a consequence the DAF 
approximation to the function is very stable in this region. As of A increases beyond the 
plateau the width of the DAF window in frequency no longer captures the. true signal and 
as a result, the true DAF signal begins to be severely averaged. In the extrenie, only the 



zero frequency remains and y^^f(;c„)= /daf and hence SJ^ofA) ^ 0. As we discuss 

below, one can usefully correlate the transition behavior with the best 
DAF-approximation. The first extremely rapid decrease is due to the fact the DAF is 
interpolating and not well tempered. It is the region beyond the initial rapid decrease that 
is important (z.e., c^A 2: 1.5). To understand the behavior in this region, we^vrnteiS^i/^^^ 
in the form 



Ylnii^fMYnAF + 2( A/(x.))daf( AIfC^h) - W) + if^U^n) ' fpArY (17) 



(p) 



N 

where f^^^i^") is the DAF approximation using a c^A in the middle of the plateau 

/ip) 
(x«) , The cross term averages to zero because the DAF 

approximation is interpolating and hence the (Af{x„))^^^ fluctuate rapidly reflecting the 
presence of noise. Thus 



Sm{<t/A) = 



1 



E.(A/(x„))^,^ + 



N 



(18) 



which decreases rapidly since E„ {AJ{x^f^j^ is positive and rapidly decreasing as the 
high frequency noise is eliminated from the signal. The transition into the plateau reflects 
a change from an interpolative to a well tempered behavior. Although the algorithm 
presented in this section only refers to periodic extensions, we stress that this is only one 
possibility out of many. 
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NUMERICAL EXAMPLES 

Two numerical examples are presented in this section to show the lisefiflness of 
our algorithm. 

Case I 
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The first one is the extrapolation ofy(x)=sin(5;D:/128), to which noise has been 
added. The Hermite DAF parameters are M=6 for padding/extension and M=l for 
smoothmg in our numerical examples. The weight W^^ was taken as discussed above. The 
values at 220 evenly spaced grid points are input over the range [0,219], with 50% 
5 random noise added (f=f x [1+ random(-0.5,0.5)]. The continuation of the solid curve 

from points JC220 to ^256 shows the function without noise. We shall predict the remaining 
36 points (excluding ^256 because the function there must equal the fimction at jcq) by the 
periodic extension algorithm presented in this paper. Because the original continuous 
function without noise is truly periodic, with period 256, this extension corresponds to 

10 filhng the gap using noisy input data. The error and the signature^' for periodic 

extension are plotted with respect to c/A in Figures 4(a) and 4(b) respectively. From 
Figure 4(b), we see that at of A = 10.5, the transition from the plateau to smoothing of the 
true signal occurs. As is evident from Figure 4(a), the minimum extension error also 
occurs at $\sigmaAbigtriangleup$ around 10.5. In Figure 3, we see the comparison of the 

1 5 true function (solid line)from the 220th to the 256th grid point, along with the periodic 

extension result ("+"* symbols). It is clearly seen that they agree very well. 

We next use the padded, extended signal for low pass filtering. We plot in Figures 
5(a) and 5(b) the L„ error and the signature of the filtered result for one complete period 
as a function of c^A using Af=12 rather than Af=6 in the DAF. This is done for 

20 convenience for reasons not germane to the subject. The result is that the of A range for 

which the DAF is well-tempered changes and the transition from denoising to signal 
modifying smoothing occurs at = 17 (Figure 5(b)). However, we see- from Figure 
5(a) that theL.minimiun error also occurs at about the same c/A, showing the robustness 
of the approach with respect to the choice of DAF parameters. In Figure 5(c), we show 

25 the resulting smoothed, denoised sine function compared to the original true signal. 

These results illxistrate the use of the DAF procedure in accurately extracting a 
band-limited function using noisy input data. Because of the relatively broad nature of 
the L„ error near the minimum, one does not need a highly precise value of c/A. 
Case II 

30 We now consider a more challenging situation. It often happens in experiments 

that the boundary condition of the experimental signal is not periodic, and is unknown, 
in general. However, the signal is approximately band-limited (i.e, in the DAF class).- 
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To test the algorithm for this case, we use the function given in Eq. (15) as an 
example. Figure 6(a) shows the function, with 20\% random noise in the range (-7,10) 
(solid line). These noisy values are assumed known at only 220 grid points in this range. 
Also plotted in Figure 6(a) are the true values of the function (dashed" line) on the 36 
5 points to be predicted. In our calculations, these are treated, of course, as unknown and 

are shown here only for reference. It is clearly seen tfiat the original function is not 
periodic on the range of 256 grid points. We force the noisy function to be periodic by 
padding the values of the function on these last 36 points, using only the known, noisy 
220 values to periodically siuround the gap. 

10 As mentioned in section IV, for nonperiodic signals, the periodi6-' extension is 

simply a scheme to provide an artificial boundary condition in a way. that- does not 
significantly corrupt the frequency distribution of the underlying true signal in the 
sampled region. The periodic padding signature is shown in Figure 7. Its behavior is 
similar to that of the truly periodic signal shown in Figure 4(b). The second rapid 

1 5 decrease begins at about ofA=5.2 and the periodic padding result for this D AF parameter 

is plotted in Figure 6(b) along with the original noisy signal. Compared with the original 
noisy signal, it is seen that the signal in the extended domain is now smoothed. In order 
to see explicitly the periodic property of the extended signal and the degree to which 
aliasing is avoided, we filter the noise out of the first 220 points using an appropriate 

20 Hermite DAF. The L„ error and the signature of the DAF-smoothed results are plotted 

in Figures 8(a) and 8(b) respectively. Again they correlate with each other very well. 
Both the minimum error and the starting point of the second rapid decrease occur at about 
£yA=9.0, which further confirms our analysis of the behavior of the signature. In Figure 
9, we present the smoothed signal (solid line) along with true signal (dashed line), 

25 without any noise added. It is seen that in general, they agree with each other very well 

in the original input signal domain. 

Application of the pseudo-signal in the extended domain clearly effectively 
avoids the troublesome aliasing problem. The minor errors observed occur in part 
because of the fact that the random noise contains not only high frequency-components 

30 but also some lower frequency components. However, the Hermite DAF is used only as 

a low pass filter here. Therefore, any low frequency noise components are left untouched 
in the resulting filtered signal. Another factor which may affect the accuracy of filtering- 
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is that we only use Af=12. According to previous analysis of the DAF theory, the higher 
the M value, the greater the accuracy [12], but at the expense of increasing the DAF 
bandwidth {dL increases). However, as AT is increased, combined with. the appropriate 
dL, the D AF-window is better able to simulate an ideal band-pass filter (while ^/i// being 
infinitely smooth and with exponential decay in physical and Fourier space.) Here we 
have chosen to employ M=12 because our purpose is simply to illustrate the use of our 
algorithm, and an extreme accuracy algorithm incorporating this principles is directly 
achievable. 

CONCLUSIONS AND DISCUSSIONS 

This paper presents a DAF-padding procedure for periodically expending a 
discrete segment of a signal (which is nonperiodic).The resulting periodic signal can be 
used in many other numerical apphcations which require periodic boundary conditions 
and/or a given number of signal samples in one period. The power of the present 
algorithm is that it essentially avoids the introduction of aliasing into the true signal. It 
is the well-tempered property of the DAFs that makes them robust computational tools 
for such applications. Application of an appropriate well-tempered DAF to the 
periodically extended signal shows that they are also excellent low pass filters. Two 
examples are presented to demonstrate the use of our algorithm. The first one is a 
truncated noisy periodic function. In this case, the extension is equivalent to an 
extrapolation. 

Our second example shows how one can perform a periodic ex-tension of a 
nonperiodic, noisy fmite-length signal. Both examples demonstrate that the algorithm 
works very well under the assimiption that the true signal is continuous and smooth. In 
order to determine the best DAF parameters, we introduce a quantity called the signature. 
It works very well both for extensions and low pass filtering. By examining the behavior 
of the signature with respect to ^A, we can determine the overall frequency distribution 
of the original noisy signal working solely in physical space, rather than having to 
transform to Fourier space. 
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GENERALIZED SYMMETRIC INTERPOLATING WAVELETS 
INTRODUCTION 

The theory of interpolating wavelets based on a subdivision scheme'has attracted 
much attention recently [1,9, 12, 13, 17, 22, 27, 29. 40, 42, 45, 46, 47, 4§, 49, 54, 55, 56, 
65 and 66], Because the digital sampling space is exactly homomorphic to the multi scale 
spaces generated by interpolating wavelets, the wavelet coefficients can be obtained from 
linear combinations of discrete samples rather than froni traditional inner product 
integrals. This parallel computational scheme significantly decreases the computational 
complexity and leads to an accurate wavelet decomposition, without any pre-conditioning 
or post-conditioning processes. Mathematically, various interpolating wavelets can be 
formulated in a biorthogonal setting. 

Following Donoho's interpolating wavelet theory [12], Harten has described a 
kind of piecewise biorthogonal wavelet construction method [17]. Swelden 
independently develops this method as the well-known "lifting scheme" [56], which can 
be regarded as a special case of the Neville filters considered in [27]. The lifting scheme 
enables one to construct custom-designed biorthogonal wavelet transforms by just 
assuming a single low-pass filter (a smooth operation) without iterations. Theoretically, 
the interpolating wavelet theory is closely related to the finite element technique in the 
numerical solution of partial differential equations, the subdivision scheme for 
interpolation and approximation, multi-grid generation and surface fitting techniques. 

A new class of generalized symmetric interpolating wavelets _(GSIW) are 
described, which are generated fi-om a generalized, window-modulated -interpolating 
shell. Taking advantage of various interpolating shells, such as Lagrange polynomials 
and the Sine fimction, etc., bell-shaped, smooth window modulation leads to wavelets 
with arbitrary smoothness in both time and frequency. Our method leads to a powerful 
and easily implemented series of interpolating wavelet. Generally, this novel designing 
technique can be extended to generate other non-interpolating inultiresolution analyses 
as well (such as the Hermite shell). Unlike the biorthogonal solution discussed in [6], we 
do not attempt to solve a system of algebraic equations explicitly. We fir-St choose an 
updating filter, and then solve the approximation problem, which is arth-order-accurate 
reconstruction of the discretization. Typically, the approximating fimctional is a 
piecewise polynomial. If we use the same reconstruction technique at all the points and- 
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at all levels of the dyadic sequence of uniform grids, the prediction will have a Toplitz- 
like structure. 

These ideas are closely related to the distributed approximating functionals 
(DAFs) used successfully in computational chemistry and physics [20; 21^ 22, 65, 66, 
67], for obtaining accurate, smooth analytical fits of potential-energy surfaces in both 
quantum and classical dynamics calculations, as v/ell as for the calculation of the state-to- 
state reaction probabilities for three-dimension (3-D) reactions. DAFs provide a 
numerical method for representing functions Icnown only on a discrete grid of points. The 
underlying function or signal (image, conunimication, system, or human response to 
some probe, etc.) can be a digital time sequence (i.e., finite in length and 1 -dimensional), 
a time and spatially varying digital sequence (including 2-D images that can -vary with 
time, 3-D digital signals resulting from seismic measurements), etc. The general stmcture 
of the DAF representation of the function,/;^^^^,/), where x can be a vector (/.e., not just 
a single variable), is 

foAfix. g=S,^^-x;|a/A)y(;c,,g 
where (t)(x-a:^.)|a/A) is the "discrete DAF'\/{Xj ,/^) is the digital value of the "signal" at 
time r^, and M and o/A will be specified in more detail below. They are adjustable DAF 
parameters, and for non-interpolative DAF, they enable one to vary the behavior of the 
above equation all the way from an interpolation limit, where 

(i.e., the DAF simply reproduces the input data on the grid to as high accuracy as desired) 
to the well-tempered limit, where 

for functionfixJp)eL\R). Thus the well-tempered DAF does not exactly reproduce the 
input data. This price is paid so that instead, a well-tempered DAF approximation makes 
the same order error off the grid as it does on the grid (i.e., there are no special points). 
We have recently shown that DAFs (both interpolating and non-interpolating) can be 
regarded as a set of scaUng functionals that can used to generate extremely robust 
wavelets and their associated biorthogonal complements, leading to a full multiresolution 
analysis [22, 46, 47, 48, 49, 54, 55, 66, 67]. DAF-wavelets can therefore seirve as an 
alternative basis for improved performance in signal and image processing. 

The DAF wavelet approach can be applied directly to treat bounded domains. As 
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shown below, the wavelet transfonn is adaptively adjusted around the boundaries of 
finite-length signals by conveniently shifting the modulated window. Thus the 
biorthogonal wavelets in the interval are obtained by using a one-sided stencil near the 
boundaries. Lagrange interpolation polynomials and band-limited Sine functionals in 
5 Paley-Wiener space are two conunonly used interpolating shells for signal approximation 

and smoothing, etc. Because of their importance in numerical analysis, we use these two 
kinds of interpolating shells to introduce our discussion. Other modulated windows, such 
as the square, triangle, B-spline and Gaussian are under study with regard to the time- 
frequency characteristics of generalized interpolating wavelets. By carefully designing 
10 the interpolating Lagrange and Sine functionals, we can obtain smooths-interpolating 

scaling functions with an arbitrary order of regularity. 

INTERPOLATING WAVELETS 

The basic characteristics of interpolating wavelets of order D discussed in 
15 reference [12] require that, the primary scaling function, satisfies the following 

conditions. 

(1) Interpolation: 



20 



where Z denotes the set of all integers. 

(2) Self-Induced Two-Scale Relation: ^ can be represented-.as a linear 
combination of dilates and translates of itself, with a weigTit given by the 
value of^at A/2. 

25 <*(x) = X<#(^/^)$^(2x-.A) (2) 

k 

This relation is only approximately satisfied for some interpolating wavelets 
discussed in the later sections. However, the approximation can be made arbitrarily 
accurate. 

(3) Polynomial Span: For an integer D>0, the collection of formal sums 
30 symbol £Q^jc-A) contains all polynomials of degree D. 

(4) Regularity: For a real V>0, <p \s Holder continuous of order V. 

(5) Localization: ^and all its derivatives through order \y] decay rapidly. 
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\^\x)\^AXl-^\x\r , Jce/?, 5>0, O^r^Kj (3) 
where [Vj represents the maximum integer that does not exceed V, 

In contrast to most commonly used wavelet transforms, the interpolating wavelet 
transfomi possesses the following characteristics: 

1. The wavelet transform coefficients are generated by the linear combination of 
signal samplings, 

Sj^,=r%2-n Wjrr^{f{T\k^i/2M (4) 

instead of the convolution of the commonly used discrete wavelet transform, such as 

Wdx)f{Vk)dx (5) 
where the scaling function, ^jjlx)=2/^^{2!x-kX and wavelet function, ^j j^{x)-2/^ i^l!x'k\ 
P/astheinterpolant2*-^^Ey(2"-^*:)4it(;c). - . • 

2. A parallel-computing mode can be easily implemented. The calculation and 
compression of coefficients does not depend on the results of other coefficients. For the 
halfband filter with length//, the calculation of each of the wavelet coefficients, Wjj^, does 
not exceed N+2 multiply/adds. 

3. For a Z)-th order differentiable function, the wavelet coefficients decay rapidly. 

4. In a mini-max sense, threshold masking and quantization are nearly optimal 
approximations for a wide variety of regularity algorithms. 

Theoretically, interpolating wavelets are closely related to the following 
functions: 



Band-limited Shannon wavelets 

The ;r band-limited fimction, ^:c)=sin(;cc)/(7u)6C' in Paley-Wiener space, 
generates the interpolating functions. Every band-limited function feL\R) can be 
reconstructed using the equation 

' 4-^ ^(x'k) (6) 
where the related wavelet function — Sinclet is defined as (see Figure!) . 
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Interpolating fundamental splines 

The fundamental polynomial spline of degree/), tf{x), whereD is an odd integer, 
has been shown by Schoenberg (1972), to be an interpolating wavelet (see Figure2). It 
is smooth with order i?=Z)-l, and its derivatives through order Z)-l decay exponentially 
5 [59]. Thus, 

rf(x)=ZkoP{k)/P{x^k) (8) 
where jf{x) is the B-spline of order D defined as 

(9) 

10 

Here U is the step function 



15 



(10) 



and {cP{k)) is a sequence that satisfies the infinite summation condition 

l,Ak)/P(n-k) = 6(n) (11) 

DeslaurierS'Dubuc functional 

20 Let D be an odd integer, D>0. There exist functions such that if has already 

been defined at all binary rationals with denominator 2^, it can be extended by polynomial 
interpolation, to all binary rationals with denominator 7!^^ , i.e. all points halftvay between 
previously defined points [9, 13], Specifically, to define the function at C>t+ 1/2)72' when 
it is ahready defined at all {kl'^}, fit a polynomial tTjj^ to the data {klT!, (k'/2/) for 

25 /:'€{2->[/:-(Z).l)/2],...,2iAH-(Z)+l)/2]}. This polynomial is unique 



This subdivision scheme defines a function which is uniformly continuous at the 
30 rationals and has a unique continuous extension; F^ is a compactly supportedr4nterval 

polynomial and is regular; It is the auto-correlation function of the Daubechies wavelet 
of order D+1. It is at least as smooth as the corresponding Daubechies wavelets (roughly- 
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twice as smooth). 

Auto-correlation shell of orthonormal wavelets 

If ^ is an orthonormal scaling function, its auto-correlation ^=^^*^ (-Oisan 

interpolating wavelet (FigureS) [40]. Its smoothness, localization and the two-scale 

relation are inherited from ^ . The auto-correlations of Haar, Lamarie-Battle, Meyer, and 

Daubechies wavelets lead to, respectively, the interpolating Schauder, interpolating 
spHne, interpolating, and Deslauriers-Dubuc wavelets. 

Lagrange half-band Alters 

Ansari, Guillemot, and Kaiser [1] used Lagrange symmetric halfband FIR filters 
to design the orthonormal wavelets that express the relation between the Lagrange 
interpolators and Daubechies wavelets [7]. Their filter corresponds to the Deslauriers- 
Dubuc wavelet of order D=7 (2Af-l), M=4. The transfer function of the halfband 
symmetric filter h is given by 

H{z) = Vi^zT{i') (13) 
where T is the trigonometric polynomial. Except for A(0)=l/2, at every even integer 
lattice point A(2n)=0, n^O, neZ. The transfer function of the symmetric FIR filter 
h{n)^h{-n\ has the form 

H{z) =112+ h{2n- l)(z'""'+ z"~') (14) 

The concept of an interpolating wavelet decomposition is similar to "algorithm 
a trous", the coimection having been found by Shensa [42], The self-induced scaling and 
interpolation conditions are the most important characteristics of interpolating wavelets. 
From the following equation 

Ax)-^An)^x^n) (15) 
and eq.(l ), the approximation to the signal is exact on the discrete sampling points, which 
does not hold in general for commonly used non-interpolating wavelets. 



GENERALIZED mXERPOLATING WAVELETS 
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Interpolating wavelets with either a Lagrange polynomial shell or Sine functional 
shell are discussed in detail. We call these kinds of window modulated wavelets 
generalized interpolating wavelets, because they are more convenient to construct, 
processing and extend to higher dimensional spaces. 
5 Generalized Lagrange Wavelets 

Three kinds of interpolating Lagrange wavelets, Halfband Lagrange wavelets, B- 
spline Lagrange wavelets and Gaussian-Lagrange DAF wavelets, are studied here as 
examples of the generalized interpolating wavelets. 

Halfband Lagrange wavelets can be regarded as extensions of the Dubuc 

10 interpolating functionals [9, 13], the auto-correlation shell wavelet analysis [40], and 

halfband filters [1], B-spUne Lagrange Wavelets are generated by a B-splirie-windowed 
Lagrange functional which increases the smoothness and localization properties of the 
simple Lagrange scaling function and its related wavelets. Lagrange Distributed 
Approximating Functionals (LDAF)-Gaussian modulated Lagrange polynomials, have 

15 been successfully applied for numerically solving various linear and nonlinear partial 

differential equations. Typical examples include DAF-simulations of 3-dimensional 
reactive quantum scattering and the solution of a 2-dimensional Navier-Stokes equation 
with non-periodic boundary conditions. In terms of a wavelet analysis, DAFs can be 
regarded as particular scaling fimctions (wavelet-DAFs) and the associated DAF- 

20 wavelets can be generated in a number of ways [20, 21, 22, 65, 66, 67]. 

Halfband Lagrange Wavelets 
A special case of halfband filters can be obtained by choosing the filter 
coefficients according to the Lagrange interpolation formula. The filter coefficients 
25 are given by 

(-I)"*A^-»n(A/+l/2-m) 

(A/-f,)!(A^ +„.|)!(2„-i) (16) 

These filters have the property of maximal flatness in Foiffier space, possessing a 
30 balance between the degree of flatness at zero fi-equency and the flatness at the- 

Nyquist fi-equency (half sampling). 

These half-band filters can be utilized to generate the interpolating wavelet 



wo 99/46731 



PCT/US99/05426 



33 

decomposition, which can be regarded as a class of the auto-correlated shell of 
orthogonal wavelets, such as the Daubechies wavelets [7]. The interpolating wavelet 
transform can also be extended to higher order cases using different Lagi^ge 
polynomials, as [40] 



The predictive interpolation can be expressed as 

^Sj (I) = f; P^., (Q)[Sj {i + 2n - 1) + 5^ (/ - 2/, + 1)] . '=2A+ 1 



(17) 



(18) 



where T is a projection and Sj is the yth layer low-pass coefficients. This projection 
relation is equivalent to the subband filter response of 

A(2«-1)=P^.,(0) (19) 
The above-mentioned interpolating wavelets can be regarded as the extension of the 
fundamental Deslauriers-Dubuc interactive sub-division scheme, which results when 
M=2. The order of the Lagrange polynomial is D=2M-\~3 (Figure 6(a)). 

It is easy to show that an increase of the Lagrange polynomial order Z), will 
introduce higher regularity for the interpolating functionals (Figure 7(a)). When i)->^+oo, 
the interpolating functional tends to a band-limited Sine function and its domain of 
definition is on the real line. The subband filters generated by Lagrange interpolating 
functionals satisfy the properties: 

(1) Interpolation: h(a})-^h(a^n)=l 

(2) Synmietry:/i(6?)=A(-^) 

(3) Vanishing Moments: /^xf^x)dx=<^p 

Donoho outlines a basic subband extension to obtain a perfect reconstruction. He 
defines the wavelet function as 

t^x)=^2xA) (20) 
The biorthogonal subband filters can be expressed as 
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h (^)=1, g(^)=e-'% S (6^)=e--ft(ty + tt) (21) 

However, the Donoho interpolating wavelets have some drawbacks. Because the low- 
pass coefficients are generated by a sampling operation only, as the decomposition layer 
increases, the correlation between low-pass coefficients become w^iiker. The 
interpolating (prediction) error (high-pass coefficients) strongly increases, which is 
deleterious to the efficient representation of the signal. Further, it can not be used to 
generate a Riesz basis iorl}{R) space, 

Swelden has provided an efficient and robust scheme [56] for constructing 
biorthogonal wavelet filters. His approach can be utilized to generate high-order 
interpolating Lagrange wavelets with higher regularity. As Figure 4 shows, Pq is the 
interpolating prediction process, and the P, filter is called the updating filter, used to 
smooth the down-sampling low-pass coefficients. If we choose Pq to be the same as P,, 
then the new interpolating subband filters can be depicted as 



h^{co) = h{o)) 

hi{o))^\^g{Q>)P{lm) . ^22) 

g,(a>) = e-'-'-A(fi))/»(2fi>) 

The newly developed filters h^yg^ h , and S also generate the biorthogonal dual pair 

for a perfect reconstruction. Examples of biorthogonal lifting wavelets with re'gularity 
D=3 are shown in Figure 5. Figure 6 gives the corresponding Fourier responses of the 
equivalent subband decomposition filters. 



B-Spline Lagrange Wavelets 

Lagrange polynomials are natural interpolating expressions for fimctional 
approximations. Utilizing a different expression for the Lagrange polynomials, we can 
construct other forms of useful interpolating wavelets as follows. We define a class of 
symmetric Lagrange interpolating fimctional shells as 



n ^ 

t.-Mj*o (23) 
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It is easy to verify that this Lagrange shell also satisfies the interpolating condition on 
discrete, integer points, 



k = 0 
otherwise 

(24) 



However, simply defining the filter response as 

A(it)=P(A/2)/2, ifc=-M, M (25) 
leads to non-stable interpolating wavelets, as shown in Figure 7. 

Including a smooth window, which vanishes at the zeros of the Lagrange 
10 polynomial, will lead to more regular interpolating wavelets and equivalent subband 

filters (as shown in Figure 7 and 8). We select a well-defined B-spline function as the 
weight window. Then the scaling fimction (mother wavelet) can be defined as an 
interpolating B-Spline Lagrange functional (BSLF) 

where N is the B-spline order and, 7 is the scaling factor to control the window width. 
To ensure coincidence of the zeroes of the B-spline and the Lagrange polynomial, we set 

2Af=7x(A^+l) (27) 

20 To ensure the interpolation condition, the B-spline envelope degree M must be odd 

number. It is easy to show that if B-spline order is N=4k^l, rj can be any odd" integer 
(2/:+l); if //is an even integer, then 77 can only be 2. When //=4^-l, wecah not construct 
an interpolating shell using the definition above. From the interpolation and self-induced 
scaling properties of the interpolating wavelets, it is easy to verify that 

25 /i(t)=jzU^2)/2,A--2A/+l,2M-l (28) 

Gaussian-Lagrange DAF Wavelets 

We can also select a class of distributed approximation functional-^Gaussian- 
Lagrange DAFs (GLDAF) as our basic scaling function to construct interpolating 
30 wavelets as: 

= ^.W ft TT (29) 
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where WJX) is a window function. It is chosen to be a Gaussian, 

= (30) 
ecause it satisfies the minimum firame bound condition in quantum physics. Here ^ris a 
window width parameter, and PJx) is the Lagrange interpolation kernel The DAF 
scaling function has been successfully introduced as an efficient and powerful grid 
method for quantum dynamical propagations [40], Using Swelden's lifting scheme [32], 
a wavelet basis is generated. The Gaussian window in our DAF-wavelets efficiently 
smoothes out the Gibbs oscillations, which plague most conventional wavelet bases. The 
following equation shows the connection between the B-spline window function and the 
Gaussian window [34]: 




for large N. As in Figure 12, if we choose the window width 

a = r}^(N+ I)/ 12 (32) 

the Gaussian-Lagrange wavelets generated by the lifting scheme will be similar to the B- 
spline Lagrange wavelets. Usually, the Gaussian-Lagrange DAF displays a slightly better 
smoothness and more rapid decay than the B-spline Lagrange wavelets. If we select more 
sophisticated window shapes, such as those popular in engineering (Bartlett, Hanning, 
Hamming, Blackman, Chebychev, andBessel windows), the Lagrange wkvelets can be 
generalized further. We shall call these extensions Bell-windowed Lagranje wavelets. 



Generalized Sine Wavelets 

As we have mentioned above, the ;rband-limited Sine function, 

<z5(^)=sin(;cc)/(;n:)(r (33) 
in Paley-Wienex space, constructs an interpolating function. Every tt band-limited 
function fEL\R) can be reconstructed by the equation 

= (34) 



where the related wavelet function — Sinclet is defmed as (see Figure!) 
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sin;r(2x-l)-sin;r(x-l/2) (35) 
^^""^ ;r(x-l/2) 

The scaling Sine function is the well-known ideal low-pass filter, which possesses the 
ideal square filter response as 

5 

^^'"^io. W2<|a.|.. (36) 

Its impulse response can be generated as 

h[k] =>f^.^) d'^d6yi^^sin{M)l{^) (37) 
1 0 The so-called half-band filter possess a non-zero impulse only at the odd hiieger sampler, 

A(2A+1), while at even integers, h[lk]=Q unless a ifc=0. 

However, this ideal low-pass filter is never used in application. Since the digital 
filter is an IIR (infinite impulse response) solution, its use as a digital cutofTFIR (finite 
impulse response) will produce Gibbs phenomenon (overshot effect) in Fourier space, 
1 5 which degrades the firequency resolution (Figure! 1). The resulting compactly supported 

Sine sealing and wavelet fimctions, as well as their biorthogonal dual scaling and wavelet 
fimctions, are shown in Figure 12. We see that the regularity of the cutoff Sine is 
obviously degraded with a firactal-like shape, which leads to poor time localization. 
B-Spline Sine Wavelets 

20 Because the ideal low-pass Sine wavelet can not be implemented "ideally" by FIR 

(finite impulse response) filters, to eliminate the cutoflf singularity, a windowed 
weighting technique is employed to adjust the time-firequency localization of the Sine 
wavelet analysis. To begin, we define a synunetric Sine interpolating fiirictional shell as 



25 />(x)=-^ (38) 



si n(/ec/2) 

7tX 



Utilizing a smooth window, which vanishes gradually at the exact zeros of the Sine 
fimctional, will lead to more regular interpolating wavelets and equivalent subband filters 
(as shown in Figure 13 and 14). For example, we illustrate using a well-defined B-spline 
30 fimction as the weight vdndow. Then the scaling fimction (mother wavelet)^ can be 

defined as an interpolating B-spline Sine fimctional (BSF) 
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^""^ ^""^ ^ (39) 

/?^(x/;7) sin(/g/2) 

where N is the B-spline order and, rj is the scaling factor to control the. window width. 
To ensure the coincidence of the zeroes of the B-spline and the Sine shell, we set 

2M+l=7x(Ar+l)/2 (40) 
To maintain the interpolation condition, A(2fc)=0, fc^O, it is easy to show that when the 
B-spIine order N='4k^ly rjmzy be any odd integer (2AH-1). If TV is an even integer, then 
7j can only be 2. When N-AkA, we can not construct interpolating shell using the 
definition above. The admissibility condition can be expressed as 

lv = 2* + l,7^ = 4/+i (41) 

From the interpolation relation 

fl.A: = (} 



0./:^() (42) 



and the self-induced two-scale relation 

(^SU) = 2;^(^/2)^(a,^^j (43) 

it is easy to show that 

20 h{k)^^Jkll)l2, k=^2M+\, 2M-\ (44) 

Gaussian-Sinc DAF Wavelets 

We can also select a class of distributed approximation fiinctionals, i.e., the 
Gaussian-Sinc DAF (GSDAF) as our basic scaling function to construct interpolating 
25 scalings, 

^^^^^^sinM (45) 



where WJ[x) is a window function which is selected as a Gaussian, 



(46) 
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Because it satisfies the minimum frame bound condition in quantum physics, it 
significantly improves the time-frequency resolution of the Windowed-Sinc wavelet. 
Here ois a window width parameter, and P{x) is the Sine interpolation kernel. This DAF 
scaling fimction has been successfully used in an efficient and powgirful^d method for 
quantum dynamical propagations [40]. Moreover, the Hermite DAF is known to be 
extremely accurate for solving the 2-D harmonic oscillator, for calculating the 
eigenftmctions and eigenvalues of the Schrodinger equation. The Gaussian window in 
our DAF-wavelets efficiently smoothes out the Gibbs oscillations, which plague most 
conventional wavelet bases. The following equation shows the connection between the 
B-spline and the Gaussian windows [34]: 



the Gaussian Sine wavelets generated by the lifting scheme will be similar to the B-spline 
Sine wavelets. Usually, the Gaussian Sine DAF displays a slightly better smoothness and 
rapid decay than the B-spline Lagrange wavelets. If we select more sophisticated wiiidow 
shapes, the Sine wavelets can be generalized further. We call these extensions Bell- 
windowed Sine wavelets. The available choices can be any of the popularly used DFT 
(discrete Fourier transform) windows, such as Bartlett, Hanning, Hamming, Blackman, 
Chebychev, and Besel windows. 

ADAPTIVE BOUNDARY ADJUSTMENT 

The above mentioned generalized interpolating wavelet is defmed on the domain 
C{R). Many engineering applications involve finite length signals, such as image and 
isolated speech segments. In general, we can define these signals on C[0,1]. One could 
set the signal equal to zero outside [0,1], but this introduces an artificial "jump" 
discontinuity at the boundaries, which is reflected in the wavelet coefficients. It v^all 
degrade the signal filtering and compression in multi scale, space. Developing wavelets 
adapted to "life on an interval" is useful. Periodization and symmetric periodization are 
two commonly used methods to reduce the effect of edges. However, unless the finite 
length signal has a large flat area around the boundaries, these two methods cannot 



for large N, As in Figure 6, if we choose the window width 




(47) 



cr=;7V(^+l)/12 



(48) 
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remove the discontinuous effects completely [4,6,1 1]. 

Dubuc utilized an iterative interpolating function, F^y on the finite interval to 
generate an interpolation on the set of dyadic rationals Dj. The interpolation in the 
neighborhood of the boundaries is treated using a boundary-adjusted functional, which 
5 leads to regularity of the same order as in the interval This avoids the discontinuity that 

results from periodization or extending by zero. It is well known that this results in 
weaker edge effects, and that no extra wavelet coefficients (to deal with the boundary) 
have to be introduced, provided the filters used are symmetric. 

We let Kj represent the number of coefficients at resolution layer 7, where KfO!, 
10 Let 2'>2Z)+2, define the non-interacting decomposition. If we let 7o-hold the non- 

interaction case 2^>2ZH2, then there exist functions ^"""^ , y/ '"""^ . , such that for 



15 



20 



all/eC[0,l], 



/ interval Iniervai 1 1 j i • t • * . 

The , ^ , are called the interval mterpolatmg scalings and wavelets, 

which satisfy the interpolation conditions 

[wTr' ( ) = -"'^^^1.- ' ^ < (50) 
The interval scaling is defined as 

^j;j^.i;>_0-l^jt<2> ^^^^ 
25 where ^^J[o.ii is called the "inner-scaling" which is identical to the fundamental 

ffiight J Uft 

interpolating function, and ^ and ^ are the "lefl-boundary" and the "right- 
boundary" scalings, respectively. Both are as smooth as ^J^k\[o^y Interval wavelets are 
defined as 
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.OSk<lD/li 

1'J.k\[0.l).lDfl]n<2J -lD/2] ^^^^ 



Left Right 

i^.*l[o.i] is the inner-wavelet, and ^ and ^ are the left and/right-boundary 

wavelets, respectively, which are of the same order regularity as the inner-wavelet [13], 
5 The corresponding factors for the Deslauriers-Dubuc interpolating wavelets are 

M=2, and the order of the Lagrange polynomial is i>=2A/-l=3, The interpolating wavelet 
transform can be extended to high order cases by two kinds of Lagrange polynomials, 
where the inner-polynomials are defined as [14] 

10 -'^ ' -ii.^(ii«-l)-(2m-l) (53) 



This kind of polynomial introduces the interpolation in the intervals according to 

(54) 



and the boundary polynomials are 



2K4-\ r-m . r. 



(55) 



20 which introduce the adjusted interpolation on the two boundaries of the intervals. That 

is, 

D 

PjSiO = i:Lj{ii2)SjUA-ld - I),/ = 2Jk + I.O S A i [d . I J/ 2 ^^gj 

25 The left boundary extrapolation outside the intervals is defined as 

PjSi'X)^ £/.j(-l/2)Sy(2J).;=:2/: + l,0^AiL^-lJ/2 _ 

joo (57) 



30 



and the right boundary extrapolation is similar to the above. The boundary adjusted 
interpolating scaling is shown in Figure 16. 

Although Dubuc shows the interpolation is almost twice differentiable, there 
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still is a discontinuity in the derivative. In this paper, a DAF-wavelet based boundary 
adjusted algorithm is introduced. This technique can produce an arbitrary smooth 
derivative approximation, because of the infinitely differentiable character of the 
Gaussian envelope. The boundary-adjusted scaling fimctionals are generated as 
5 conveniently as possible just by window shifting and satisfy the following equation. 

^Jix)=W{x'lm)P{xl ?w=l(M-l)/2j, L(M.l)/2j (58) 
where ^Jx) represents different boundary scalings, W{x) is the generalized window 
function and P{x) is the symmetric interpolating shell. When /w>0, left boundary 
fimctionals are generated; when m<0, we obtain right boundary fimctionals. The case 

10 m-0 represents the inner scalings mentioned above. 

One example for a Sinc-DAF wavelet is shown in Figure 17. We choose the 
compactly-supported length of the scaling function to be the same as the halfband 
lagrange wavelet. It is easy to show that our newly developed boundary scaUng is 
smoother than the commonly used Dubuc boundary interpolating fimctional. Thus it will 

15 generate a more stable boundary adjusted representation for finite-length wavelet 

transforms, as well as a better derivative approximation around the boundaries. Figurel 8 
is the boundary filter response comparison between the halfband Lagrange wavelet and 
our DAF wavelet. It is easy to establish that our boundary response decreases the 
overshoot of the low-pass band filter, and so is more stable for boundary frequency 

20 analysis and approximation. 

APPLICATIONS OF GSIWs 

Eigenvalue Solution of 2D Quantum Harmonic Oscillator 
As discussed in [22], a standard eigenvalue problem of the Schrodinger 
25 equation is that of the 2D harmonic oscillator, 



^2 2 q2 "I 



Here 0^ and Ej^ are the kth eigenfimction and eigenvalue respectively. The" eigenvalues 
30 are given exactly by 

^A,.*: = > + , 0£/:<oo, 0^|<jt2 ' (60) 

with a degeneracy (k^~k+ 1 ) in each energy level Ef^= 1 +/:. The 2D version of the wavelet' 
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DAF representation of the Hamiltonian operator was constructed and the first 21 
eigenvalues and eigenfunctions obtained by subsequent numerical diagonalization of the 
discrete Sinc-D AF Hamiltonian, As shown in Tab. 1 , all results are accurate to at least 1 0 
significant figures for the first 16 eigenstates. It is evident that DAF-wavelets are 
powerful for solving eigenvalue problems. 

Tablel 



Eigenvalues of the 2D harmonic oscillator 



k=k+K 




Exact Solution 


Sinc-DAF Calculation 


0 


1 


1 


0.99999999999835 


1 


2 


2 


1.99999999999952 
1.99999999999965 


2 


3 


3 


2 99999999999896 
2.99999999999838 
2.99999999999997 


3 


4 


4 


3.99999999999943 
3.99999999999947 
3.99999999999986 
3.99999999999994 


4 


5 


5 


4.99999999999907 
4.99999999999953 
4.99999999999989 
5.00000000000674 
5.00000000000813 


5 


6 


6 


5.99999999999982 
6.00000000000018 
6.00000000000752- 
6.00000000000801 . 
6.0000000001 1972_ 
6.00000000012005 ■ 



Target Extraction 

Military target extraction, including such applications as high-resolution radar 
(aerial photograph) imaging, radar echo, and remote sense detection, is a difficult 
subject. National Defense Agencies (e.g. the Navy, Army, and Air Force) have great 
interest in technical advances for reconnaissance, earlier warning and target 
recognition. Some of our DAF-v/avelet schemes represent a significant breakthrough 
for these very difficult tasks. Compared with other methods, our algorithm possesses a 
very high resolution for target localization and high efficiency for clutter/noise 
suppression, as well as computational efficiency. 
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Detecting a military target in a low luminance environment is challenging 
work for image processing. To improve the target discrimination, the visibility of 
differences between a pair of images is important for modem image restoration and 
enhancement. We construct a method for detectability using a multiple channel 

5 enhancement technique. The images were captured on a color monitor at a viewing 

distance giving 95 pixels per degree of visual angle and an image size of 5.33*5.05 
deg. The mean luminance of the images was about lOcd/m^ [39]. Using our newly 
developed visual enhancement techniques, visual targets can be extracted very 
accurately in a low-luminance environment for detection and warning. The technique 

0 combines the response of human vision system (HVS) with multiresolutibn 

enhancement and restoration methods. The simulation of tank-target detection in a 
low-luminance environment is shown in Figure 19. 
Image Filtering 

Image de-noising is a difficult problem for signal processing. Due to the 

5 complicated structure of image and background noise, an optimal filtering technique 

does not currently exist. Generally, the possible noise sources include photoelectric 
exchange, photo spots, image communication error, etc. Such noise causes the visual 
perception to generate speckles, blips, ripples, bumps, ringing and aliasing. The noise 
distortion not only affects the visual quahty of images, but also degrades the 

0 efficiency of data compression and coding. De-noising and smoothing are extremely 

important for image processing. 

We use a DAF-wavelet to generate a more efficient, human-vision-system- 
based image processing technique, which processes the advantages of 1) long range 
de-correlation for convenience of compression and filtering; 2) high perceptual 

5 sensitivity and robustness; 3) filtering that takes accoimt of the human visual 

response. It therefore can enhance the most important visual information, such as 
edges, while suppressing the large scale of flat regions and backgroimd; 4) it can be 
carried out with real-time processing. 

Biorthogonai interpolating wavelets and corresponding filters are constructed 

0 based on Gauss-Lagrange distributed approximating fiinctionals (DAFs). The-utility 

of these DAF wavelets and filters is tested for digital image de-noising in combination 
with a novel blind restoration technique. This takes account of the response of human - 
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vision system so as to remove the perceptual redundancy and obtain better visual 
performance in image processing. The test results for a color photo are shown in 
Figure20. It is evident that our Color Visual Group Normalization technique yields 
excellent contrast and edge-preservation and provides a natural color result for the 
restored image [48]. 

Imaging Enhancement 

Mammograms are complex in appearance and signs of early disease are often 
small and/or subtle. Digital manunogram image enhancement is particularly important 
for solving storage and logistics problems, and for the possible developnrent of an 
automated-detection expert system. The DAF-wavelet based mammogran - 
enhancement is implemented in the following manner. First we generate a perceptual 
lossless quantization matrix Qj„ to adjust the original transfomi coefficients Cjjk). 
This treatment provides a simple human- vision-based threshold technique for the 
restoration of the most important perceptual information in an image. For grayscale 
image contrast stretching, we appropriately normalize the decomposition coefficients 
according to the length scale, L, of the display device [16] so that they fall in the 
interval of [0,1] of the device frame 



where the constant and function ^ are appropriately chosen so that the desired 
portion of the grayscale gradient is stretched or compressed. 

To test our new approach, low-contrast and low quality breast mammogram 
images are employed. A typical low quality front-view image is shown in Figure 
21(a). The original image is coded at 512x512 pixel size with 2 bytes/pixel and 12 
bits of gray scale. We have applied our edge enhancement normalization and device- 
adapted visual group normalization. As shown in Figure 2.1(b), and Figure21(c), there 
is a significant improvement in both the edge representation and image contrast. In 
particular, the domain and internal structure of high-density cancer tissues are niore 
clearly displayed. Figure 22(a) is an original 1024x1024 side-view breast image 
which has been digitized to a 200 micron pixel edge with 8 bits of gray scale. The 




(61) 



(62) 
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enhanced image result is shown in Figure 22(b). In this case we again obtain a 
significant improvement in image quality as described herein. 

CONCLUSION 

In summary, a new class of wavelets — generalized symmetric interpolating 
wavelets were described, which are generated by a window modulated interpolating 
shell. Due to the absence of a complicated factorization process, this kind of 
interpolating wavelet is easily implemented and possesses very good characteristics in 
both time (space) and spectral domains. The stable boundary adjustment can be 
generated by a window shifting operation only. It overcomes the overshoot of the 
boundary response introduced by other boundary processing, such as Dubiic Lagrange 
wavelet and Daubechies boundary fihers. Many successful applications of DAF- 
wavelets have been reported to illustrate its practicality and its mathematical behavior. 
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DUAL WINDOW SELECTIVE AVERAGING FTT.TER 
INTRODUCTION 

Local averaging is one of the simplest filtering techniques. It preserves the 
mean gray level while suppressing the variability in flat regions. However, simple 
linear local averaging is undesirable for image smoothing because it is incapable of 
preserving image edges. Linear local averaging filters are essentially low pass filters, 
which tend to blur the edges and fme structures in the original image. In order to 
preserve the edges while achieving some degree of smoothing, it is desirable to 
employ nonlinear filters, A number of nonlinear selective averaging methods [1-8] 
have been devised for this purpose. The basic idea for these methods is to select only 
a portion of the gray level values in the local window to use in a (weighted) average. 
In references [3-5], an alpha-trimmed mean filter (a-TMF), which uses a "median 
basket" to select a predetermined number of pixels above and below the median pixel 
to the sorted pixels of the moving window, was proposed. 

The values in the basket are averaged to give the a-TMF filtering output. An 
asymmetric way to select the averaging pixels whose values are close to that of the 
median pixel was presented in references [4-6] and was named the modified trimmed 
mean filter (MTMF) in reference[4]. Recently, we developed a generalization of the 
a-TMF, which we denoted as GTMF [8]. 

It employs the same way to select the pixels firom the window for inclusion in 
the median basket. However, the selected pixels and the center pixel in the window 
are weighted and averaged to give the filtering output. It has been i5hown-{8] that the 
GTMF performs better than other well known filters for the removal of either 
impulse noise or additive noise. 

A new nonlinear filtering technique is disclosed, called the ''dual window 
selective averaging filter" (DWSAF), to remove additive noise (e.g., Gaussian noise). 
Assuming implicitly that the ideal image is piecewise flat, two normal concentric 
moving windows and a pixel container are employed to determine the values to be 
used in replacing the gray level value of the center pixel 7^. Three steps are employed 
in this filtering algorithm. First, the GTMF is implemented within the smaller- window 

to give an intermediate output at the center pixel of the window. Second, only 
the gray level values close to are selected from the larger window and put to the- 
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container C. Third, if the number of pixels in C equals zero, the final DWSAF output 

is just G^, otherwise, the gray level values in C are averaged to give the final 
DWSAF output D^. . - 

In contrast to the a-TMF, the GTMF also includes the center pixel /, in the 
averaging operation and its weight is usually larger than those of other pixels in the 
median basket, which is important for the removal of additive noise. 

A threshold T based on the GTMF output is used to select the pixels for 
the container. A pixel is selected for inclusion in the container if its value is in the 
range of [G,-r, G^+T]. When r=0, the DWSAF is equivalent to the GTMF within the 
smaller window, and when r= 255 (for an 8bpp gray scale image), it becomes a 
simple moving average within the larger window. 

GENERALIZED ALPHA-TRIMMED MEAN FILTER 

The implementation of the generalized trimmed mean filter (GTMF) has been 
described in detail in [17]. The pixels { /„ 4 . . ., 4.,, 4, 4^,, .../„} in the local 
window associated with a pixel 4 have been arranged in an ascending (or descending) 
order, with 4 being the median pixel. The key generalization to median filtering 
introduced in the alpha-trimmed mean filter (a-TMF) [18] is to design a median 
basket to combine a group of pixels whose gray level values are close to the median 
value of the window. 

An averaging operation is then utilized to generate an adjusted replacement 
for/^. For example, a 3-entry median basket a-TMF is implemented according to 

3 

It is evident that a single-entry median basket a-TMF is equivalent to the 
median filter and a /j-entry median basket a-TMF is equivalent to the simple moving 
average filter. 

In general, the a-TMF outperforms the median filter. However, it is still not 
optimal when filtering either the additive noise corrupted images or highly impulse 
noise corrupted images. For example, when removing additive noise, the a-TMF 
does not take the as a special pixel. As is well known, for an image corrupted by 
additive noise, has the largest probability of being the Closest to the true value 
among all the pixels in the window. Neglecting the influence of the center pixel is a 
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mistake if one desires to filter additive noise. Another difficulty for the a-TMF, 
when it is used to remove impulse noise from highly corrupted images, is that the 
pixels selected for the median basket may also be corrupted by impulse noise. It is 
therefore unreasonable for all the pixels in the basket to have the same weight. 

The GTMF uses a median basket to collect a group of pixels from 5ie* sorted 
pixel values of the window associated with 4, in the same way as the a-TMF. 

The values of the selected pixels and 4 are then weighted and averaged to give 
the GTMF output. 

For a 3-entry median basket, the output of the GTMF is given by 

tOl + 11^2 + + l«c ' (2) 

where is the GTMF output, w„ W3, and are the weights. It is interesting to 
see that when w^=0 and >v,=W2=W3= ''O, the GTMF reduces to the a-TMF. When 
w,=M/2=>V3= =0, it becomes the standard median filter. Since the GTMF takes the 
center pixel into account, we expect that it should also work better for removing 
additive noise. 

DUAL WINDOW SELECTIVE AVERAGING FILTER 

In this section, we will discuss how to implement our dual window selective 
averaging filter (DWSAF), based on the GTMF. The standard selective averaging 
algorithm [2] is not optimal because it only computes the average (or weighted- 
average) of the gray level values close to that of the center pixel in a window. 
However, sometimes it is unreasonable to choose the gray level value of the center 
pixel as a criteria because the pixel may be highly corrupted. Using the standard 
selective averaging algorithm to treat such a pixel will result in it remaining highly 
corrupted. 

The key idea of the DWSAF is to find an alternative criteria for the standard 
selective averaging algorithm to use. In reference [4], a modified trimmed mean 
filter (MTMF) was proposed to average only those pixels whose gray level values fall 
within the range [Af^-q, M^+q], where is the median value (M,=4) and ^ is a 
preselected threshold. 

Here, we propose a new filtering algorithm which employs two different sizes . 
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of concentric moving windows and W^y and a pixel container C. Three steps are 
involved in the filtering technique. First, the GTMF is employed in the smaller 
window Ws according to Eq. (2) to give an intermediate output G^, which -is used as 
the criteria for the standard selective averaging algorithm. 
5 Second, from the larger window only the pixels with gray level values 

close to are selected and placed in a container C. In doing so, a switching threshold 
ris employed to determine if a pixel in is close enough to G^. We determine if a 
pixel in belongs to C as follows: 

10 /jfcGC, if |/jfc-(?c|<T. (3) 

where 7 is the threshold. 

Third, if the number of pixels A'^in the container C is greater than zero, the 
gray level values of the pixels in the container are averaged with the same weight for 
all pixels to give the final output of the DWSAF. If A^=0, the final output equals G^.. 

1 5 We summarize this as 



Gey if i\r = 0 ' ("^^ 



where is the output of the DWSAF to replace and is the average of the gray 
20 level values in the container. The reason for employing two windows is as follows. As 

in the case of median filter, the implementation of the GTMF using a larger window 
blurs the image more than using a smaller window. However, the output of the 
GTMF in a smaller window can be used as a criteria to improve the filtering result by 
use of the standard selective averaging algorithm in the larger window. 
25 This reduces the image blurring because we only average those gray level 

values that are close to the GTMF output. For different choices of the threshold 7, the 
DWSAF can be made to be either a GTMF with respect to the smaller wrindow (T 
= 0) or a simple moving average within the larger window Wj^ (7=255 for an 8bpp 
gray-scale image). 

30 The DWSAF algorithm can be employed iteratively, which generally 

improves its performance. If the weight of the center pixel is too high, the output 
values at some pixels may remain close to their original input values after a number of 
iterations, and thus some isolated impulse-like noise may be present. Numerical 
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experimentation shows that changing the threshold Tto zero after a number of 
iterations greatly alleviates this problem. 

NUMERICAL EXPERIMENTS 

Two numerical examples are presented here for testing our filtering algorithm. 
The first one is a standard one-dimensional blocked signal, which is shown in Figure 
1(a). We degrade it with Gaussian noise of mean square error MSE=1. 00 and mean 
average error MAE=0.80 (see Figure 2(b)). The sizes of the two moving windows we 
used are 7 and 19 for and fF^ respectively. The weights for the 3-entry basket 
which we used for the GTMF are w^=wf-0, ^2=3 and >v,=4. The switching threshold 
is chosen to be r=L5, To obtain improved filtering results, the DWSAF algorithm is 
implemented recursively. 

In Figure 1(c), we show the filtered signal. It is seen fi-om Figure 1(c) that the 
filtered result is in excellent agreement with the original noise-fi-ee signal. The MSB 
and MAE of our filtered result are 6.19E-2 and 6.49E-3 respectively! The DWSAF is 
extremely efficient and robust for removing additive noise fi-om blocked signals. 

To fiirther confirm the usefulness of the algorithm, we next consider a 
two-dimensional benchmark 8 bit, 512 \times 512, "Lena" image, which has been 
degraded with two different amounts of Gaussian noise. Figure 2(a) shows the 
corrupted Lena image with peak signal-to-noise ratio PSNR=22.17 dB and Figure 
3(a) shows the corrupted image with PSNR=18.82 dB. Since the original hoise-fi-ee 
image is not exactly piecewise flat, it is not desirable to produce a filtering 
performance as efficient as in the first example. However, we expect that tiie 
DWSAF will yield improved results. 

The sizes of the two windows for both corrupted images are 3 for and 5 for 
W^. Three-entry median basket is used for both corrupted images, and the weights are 
w^:w2:wyW^=^l : 1 :1 : 10 for the image shown in Figure 2(a) and w^:w2:wyW^='l : 1 : 1 :50 
for the image shown in Figure 3(a). The initial switching threshold T is 50 for both 
corrupted images, and is changed to 0 after the first iteration for the lower noise 
image and after the second iteration for the higher noise image. Figures 2(b) and 3(b) 
show the filtered images produced by our numerical algorithm. Comparing them with 
the degraded images, it is clear that the algorithm can effectively remove noise and 
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significantly preserve edges simultaneously. For a quantitative evaluation of the 
performance of the algorithm, the PSNR, mean square error (MSE) and mean 
absolute error (MAE) comparison for different filtering algorithms are listed in 
TABLE 1. 

TABLE L 

Comparative Filtering Results for Lena Image 
Corrupted with Different Amount of Gaussian Noise 



10 



15 





Noisy Image I 


Noisy Image II 


Algorithm* 


PSNR 


MSE 


MAE 


PSNR 


MSB 


MAE 


No Denoising 


22.17 dB 


394.37 


15.97 


18.82 dB 


853.07 


23.71 


Median (3 x 3) 


29.38 dB 


75.09 


6.26 


27.60 dB 


112.97 


7.72 


Median (5 x 5) 


28.60 dB 


89.66 


6.53 


27.39 dB 


118.41 


7.53 


o-TMF (3 X 3) 


29.84 dB 


67.46 


6.07 


28.13 dB 


100.10 


7.04 


MTMF (3 X 3) 


29.91 dB 


66.46 


5.79 


28.23 dB 


97.74 


7.39 


New Approach 


30.69 dB 


55.50 


5.34 


28.96 dB 


82.69 


6.45 



20 



25 



All results are implemented recursively for optimal PSNR performance. 
The threshold used for the MTMF algorithm is optimized to q=65 for 
image I and q=100 for image II. 

The PSNR is increased about 8.52 dB for the lower noise image kid 1 0. 14 dB 
for the higher noise image, both of which are better than the best median,_a-TMF and 
MTMF filtering results. 



30 



CONCLUSIONS 

In this paper, a new filtering algorithm for removing additive noise from a 
corrupted image is presented. In comparison to standard filtering schemes, the 
DWS AF algorithm employs two concentric moving wmdows to determine the gray 
level value used to replace that of the center pixel Employing the larger -window 
does not lead to significant blurring of the image because of the effect of the smaller 
window. The GTMF is used in the smaller window for obtaining the criterion for 
standard selective averaging in the larger window. Numerical examples show that the 
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proposed filtering algorithm is both efficient and robust for recovering blocked 
signals contaminated with additive noise. 

REFERENCES 

5 LA. Scher, F. R. D. Velasco, and A. Rosenfeld, ''Some new image smoothing 

techniques," IEEE. Trans. Syst. Man, Cybem., Vol. SMC-10, pp. 153-158, 1980. 

2 A. Rosenfeld and A. C. Kak, Digital Picture Processing, New York: Academic 
Press, 1982, Vol. 1. 

3 J. B. Bednar and T. L. Watt, ''Alpha-trimmed means and their relationship to 
10 median filters," IEEE Trans. Acoust., Speech, and Signal Processing, Vol. ASSP-32, 

pp. 145-153, 1984. 

4 Y. H. Lee, S. A. Kassam, "Generalized median filtering and related nonlinear 
filtering techniques," IEEE Trans. Acoust., Speech, and Signal Processing, Vol. 
ASSP-33, pp. 672-683, 1985. 

15 5 S. R. Peterson, Y. H. Lee, and S. A. Kassam, "Some statistical properties of 

alpha-trimmed mean and standard type M filters," IEEE Trans. Acoust., Speech, and 
Signal Processing, Vol ASSP-36, pp. 707-713, 1988. 

6 P. K. Sinha and Q; H. Hong, "An improved median filter", IEEE Trans. 
Medical Imaging, Vol. 9, pp. 345-346, 1990. 
20 7 X. You and G. Grebbin, "A robust adaptive estimator for filtering noise in 

images", IEEE Trans. Image Processing, Vol, 4, pp. 693-699, 1995. 
8 D. S. Zhang, Z. Shi, D. J. Kouri, D. K. Hoffinan, "A new nonlinear image 
filtering technique," Optical Engr., Submitted. 



25 



wo 99/46731 



PCT/US99/05426 



59 

ARBITRARY DIMENSION EXTENSION OF SYMMETRIC DAF 
INTRODUCTION 

The distributed approximating functional (DAF) approach to the time 
evolution of wave packets is a powerful, new method aimed at taking advantage of the 
5 local nature of the potential in the coordinate representation for ordinary chemical 

collisions, and the localized nature of the kinetic energy in this same representation. 
These features of the Hamiltonian lead to a highly band-pass expression for the 
propagator, specific to a restricted class of wave packets (DAFs). With control of the 
parameters which determine it, the DAF class can be made to include the wave 

10 packets of interest in any particular time-dependent quantum problem. Because the 

propagator is banded, its application to the propagation on the grids (1) scales like the 
number of grid points, N, in any dimension (which is the ultimate for any grid 
methods; the scaling constant depends on the band width); (2) requires reduces 
communication time when implemented on massively parallel computer; and (3) 

15 minimizes the storage requirement for the propagator. 

In general, DAF can be regarded as a radial interpolating basis, as well as the 
scaling fimction. The corresponding DAF wavelets can be implemented using 
generalized DAFs. As a kind of radial basis functional (RBF), DAF neural networks 
are constructed and applied in signal processing. 

20 We presented the derivation of the DAFs independent of the choice of 

coordinates and show how the Hermite functions becomes generalized for arbifrary 
niunbers of degrees of freedom and choice fo orthogonal coordinates. We.discuss the 
properties of the generalized "DAF polyrhonals" associate with different coordinate 
and numbers of degrees of freedom, and how angular momentum conservation leads 

25 to a radial propagator for spherical waves. Symmetric quincimx-DAF Dirichlet 

classes are carefully designed for hyper spherical sxuface representation. 

DAF THEORY 

The theory of fundamental distributed approximating functional is based on 
30 the Hermite function representation of the ID function which is 

(!)• 
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The function //2„ is the Hennite polynomial of even order, 2n. The quantity a 
is the width of the Gaussian window of the Hermite polynomial. The qualitative 
behavior of one particular Hermite DAF is shown in Figure 1, The Hermite 
polynomial His generated by the usual recursion 

fl, /2 = 0 
2x, n = l 



lxH„.,{x)-l{n-\)H„.,{x\ n>\ 



The predominant advantage of the Hennite polynomial approximation is its 
high-order derivative preservation (which leads to a smooth approximation). This 

10 paper expands and tests the approach of Madan and Milne (1994) for pricing 

contingent claims as elements of a separable Hilbert space. We specialize the-Hilbert 
space basis to the family of Hermite polynomials and use the model to price options 
on Eurodollar futures. Restrictions, on the prices of Hermite polynomial risk for 
contingent claims with different times to maturity are derived. These restrictions are 

1 5 rejected by our empirical tests of a four-parameter model. The unrestricted results 

indicate skewness and excess kurtosis in the implied risk-neutral density. These 
characteristics of the density are also mirrored in the statistical density estimated from 
a time series on LIBOR. The out-of-sample performance of the four-parameter model 
is consistently better than that of a two-parameter version of the model. 

20 The Hermite polynomial are given in terms of their generator by 



if,.(^)exp(-x^) = (-1)" |ir«>^P(-^') = (-1)"^' ^'^P^-^'^ 



(3) 



where V^is the x component of the gradient operator. This equation provides two 
25 equivalent and useful ways in which the above function can be expressed as 

Six - A cj) = -^t.^^y W-^' r V^' expKx - X') V la' ] 

n«0 (4) 

or 



^f(x-x'la) = ^|;(l!-ri[2c7^rvJrexpH^-x')V2o-^] 



(5)- 
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where in the second equation V!" has been replaced by • The two are 

equivalent because the derivative acts on an even function of (x-x^. 

The simple practical implementation is truncating the sum at some' maximum 
value M/2 to obtain 



as indicated by the notation, the distributed approximating functional 5j^(x|cr) depend 
upon the M and a, nut it does not depend upon the coordinate system per se. Here it is 
easy to show that 



for any fixed M. The availability of two independent parameters, either of which can 
be used to generate the identity kernel or Dirac delta function, can be viewed as the 
source of robustness of the DAFs as computational tools. See e.g., D. K. Hoffinan, D. 
J. Kouri, "Distributed approximating functional theory for an arbitrary number of 
particles in a coordinate system-independent formalism," J. Physical Chemistry, 
Vol.97, No.l9, pp.4984-4988, 1993. A simple expression of Hermite DAF is as 



ARBITRARY DAF EXTENSION IN MULTI-DIMENSION SPACE 
Tensor-product extension 

Different selections of interpolating shells result in different DAFs. 
Theoretically, this kind of functional can be regarded as the smoothing operator or the 
scaling function in wavelet theory. It can be used to generate the corresponding 
wavelets (differential fiinctionals) for signal analysis. The discrete wavelet transform 
is implemented using filterbanks. Hoffinan and Kouri gave a tensor-product extension 
of DAF in multi-dimensional Hilbert space. It can be explained in detail as following. 
Consider a Dirichlet system which can be represented in a general configuration space 
by a vector R with an arbitrary number N of Cartesian components, x„ jCj, . . . By 




(6) 



Iim(J^(x-x*|o-)-<5(x-x') 



(7) 




(8) 
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"Cartesian" we mean that the "tensor" volume element for the space is given by 

y ' (9) 

5 Thus, an arbitrary function, in Hibert space can be represented by 

¥{X)-l S(X-X') '/iX')dX' (10) 
where ^X-X') is the <5'function in TV-dimension space which can be written in the 
form 

10 >■ (11) 

Using the tensor product to substitute the ID scalar variable, we can obtain the 
expression in N-dimensional Hilbert space as 



15 



20 



30 



Six - X'\ a) = (a^r^f^i^r U2ct'YVI" expR^ - X')' /2a'] 

"=0 (12) 



where the expression 



Z()=EZ-20 

"=0 ij,«iO«j«0 H^. 



(13) 



and the multi-variable 



>i (14) 

25 In multi-dimensional Hilbert space 

N 

" = Z"; (15) 

The gradient operation is 



y2„ ^ d"' ci"' 



(16) 



'A- 
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where V] is the N-dimensional Laplacian operator. 

^ ""'^ 07] 

5 The advantage of this expression is that it is independent of the particular 

choice of coordinate system and we can use whatever coordinate are convenient for a 
particular problem (e.g. Cartesian, cylindrical polar, spherical polar, hyperspherical, 
elliptical, etc., as the case might be). A 2D Tensor based Hermite DAF is shown in 
Figure2. The generalized interpolating matrix is hyper-square and can be separated by 
10 different directional vector. 



Tensor-product extension 

DAF define a class of symmetric interpolants. Besides the tensor-product 
extension, it could be with more interesting shapes. To preserve the symmetry 
15 property, any DAF or DAF-like inerpolaitng functional is expressed as 

5(x)=5(|x|) (18) 
Therefore the fundamental variable for DAF processing is |x|. The extended 
norm-preserved quincunx solution is implemented by the following substitution in N- 
D space 

20 JiL 

l^N(Zlx,r)^'^ 

(19) 

As p=q, the N-D space multi- variable becomes to the l^norm 
25 \\X\\,=(Z\^j\''y"' 



30 



(20) 



When p equals to Yi, the contour of the 2D quincunx DAF is a pseudo-diamond shape. 
The practical truncated shape is represented by the following formula: 

Kx,j/) = e-<W*bi) 

(21) 

M<x,;;) = e-<W^M)'/(2<r') 

(22)' 
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K^,y) = e-<'^*>^J'<^-') (23) 

(24) 



5 CONCLUSION 

A new algorithm for impulse noise removal was presented. A group of 
significant values in the neighboring window of one pixel are bundled and weighted 
to obtain a modified luminance estimation (MLE), A threshold selective-pass 
technique is employed to determine whether any given pixel should be replaced by its 
10 MLE. Iterative processing improves the performance of our algorithm for -highly 

corrupted images. Numerical experiments show that our technique is extremely 
robust and efficient to implement, and leads to significant improvement over other 
well-known methods. 



15 
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LAGRANGE WAVELETS FOR SIGNAL PROCESSING 
INTRODUCTION 

The theory of interpolating wavelets based on a subdivision scheme has 
attracted much attention recently. It possesses the attractive characteristic that the 
wavelet coefficients are obtained from linear combinations of discrete samples rather 
than from traditional inner product integrals. Mathematically, various interpolating 
wavelets can be formulated in a biorthogonal setting. Harten has described a kind of 
piecewise biorthogonal wavelet constmction method [12], Swelden independently has 
developed essentially this method into the well known "Ufting scheme" theory [32], 
which can be regarded as a special case of the Neville filters [19]. Unlike the previous 
method for constructing biorthogonal wavelets, which relies on explicit solution of 
coupled algebraic equations [10], the lifting scheme enables one to construct a 
custom-designed biorthogonal wavelet transforms assuming only a single low-pass 
filter without iterations. 

Generally speaking, the lifting-interpolating wavelet theory is closely related 
to: the finite element technique for the numerical solution of partial differential 
equations, the subdivision scheme for interpolation and approximation, multi-grid 
generation and surface fitting techniques. The most attractive feature of the approach 
is that discrete samplings are made which all identical to wavelet multiresolution 
analysis. Without any of the pre-conditioning or post-conditioning processes required 
for accurate wavelet analysis, the interpolating wavelet coefficients can be 
implemented using a parallel computational scheme. 

Lagrange interpolation polynomials are commonly used for signal 
approximation and smoothing, etc. By carefiiUy designing the interpolating Lagrange 
fiinctionals, one can obtain smooth interpolating scaling fimctions with arbitrary order 
of regularity. In this paper, we will present three different kinds of biorthogonal 
interpolating Lagrange wavelets (Halfband Lagrange wavelets, B-spline Lagrange 
wavelets and Gaussian-Lagrange DAF wavelets) as specific examples of generalized 
interpolating Lagrange wavelets. 

Halfband Lagrange wavelets can be regarded as an extension of Dubuc 
interpolating fimctionals [8, 11], auto-correlation shell wavelet analysis [26], and 
halfband filters [1]. B-spline Lagrange Wavelets are generated by a B-spline- 
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windowed Lagrange functional which increases the smoothness and localization 
properties of the simple Lagrange scaling function and related wavelets. 

Lagrange Distributed Approximating Functionals (LDAF)-Gaussian 
modulated Lagrange polynomials have been successfully applied for numerically 
solving various linear and nonlinear partial differential equations [40]. Typical 
examples include DAF-simulations of 3-dimensional reactive quantum scattering and 
2-dimensional Navier-Stokes fluid flow with non-periodic boundary conditions. In 
terms of wavelet analysis, DAFs can be regarded as particular scaling functions 
(wavelet-DAFs); the associated DAF-wavelets can be generated in a number of ways 
[41]. Both DAFs and DAF-wavelets are smooth and decay rapidly in both the time 
and frequency representations. One objective of the present work is to extend the 
DAF approach to signal and image processing by constructing new biorthogonal 
DAF-wavelets and their associated DAF-filters using the lifting scheme [32]. 

As an example application of Lagrange wavelets, we consider image 
processing, de-noising and restoration. This application requires dealing with huge 
data sets, complicated space- frequency distributions and complex perceptual 
dependent characteristics. De-noising and restoration play an important role in image 
processing. Noise distortion not only affects the visual quality of images, but also 
degrades the efficiency of data compression and coding. 

To explicit the time-frequency characteristics of wavelets, an earlier group 
normalization (GN) technique [28, 29] has been utilized to re-scale the magnitudes of 
various subband filters and obtain normalized equivalent decomposition filters 
(EDFs). The group nonnalization process corrects the drawback that the magnitudes 
of the transform coefficients do not correctly reflect the true strength of the various 
signal components. 

Moreover, in order to achieve the best noise-removing efficiency, the human 
vision response is best accoxmted for by a perceptual normalization {PN) based on the 
response property of the Human Vision System (HVS). The concept of visual loss-less 
quantization, introduced by Watson [39], is utilized to construct the visuaUoss-less 
matrix, which modifies the magnitudes of the normalized wavelet coefficients. 

Perceptual signal processing has the potential of overcoming the limits of the 
traditional Shannon Rate-distortion (R-D) theory for perception-dependent 
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information, such as images and acoustic signals. Previously, Ramchandran, Vetterii, 
Xiong, Heriey, Asai, and Orchard have utilized a rate-distortion compromise for 
image compression [14, 23, 24, and 42]. Our recently derived Visual<3noup 
Normalization (VGN) technique [31] can likely be used with rate-distortion 
compromise to generate a so-called Visual Rate-Distortion (VR-D) theory to further 
improve image processing. 

Softer Logic Masking (SLM) is an adjusted de-noising technique [29], 
designed to improve the filtering performance of Donoho*s Soft Threshold (ST) 
method [9]. The SLM technique efficiently preserves important information, 
particularly at an edge transition, in a manner particularly suited to human visual 
perception. 

INTERPOLATING WAVELETS 

The basic characteristics of interpolating wavelets of order D discussed in 
reference [10] require that the primary scaling function, ^, satisfies the following 
conditions. 

(1) Interpolation: 



(2) Self-induced Two-Scale Relation: ^ can be represented as a linear 
combination of dilates and translates of itself, while the weight is the valut of ^ at a 
subdivision integer of order 2. 



This is only approximately satisfied for some of the interpolating wavelets discussed 
in the later sections; however, the approximation can be made arbitrarily accurate. 

(3) Polynomial Span: For an integer 0, the collection of formal sums, 
symbolized by YJ^kA^'^y contains all polynomials of degree D, 



(5) Localization: ^and all its derivatives through order \y\ decay rapidly. 




(4) Regularity: For real V>Q, <f> is Holder continuous of order V, 
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l5d<'>(x)|<^,(l+|jc|r,x6i?.5>0,0<riKj. (3) (3) 

where [Vj represents the maximum integer which does not exceed V, 

Interpolating wavelets are particularly efficient for signal representation since 
5 their multiresolution analysis can be simply realized by discrete sampling. This makes 

it easy to generate a subband decomposition of the signal without requiring tedious 
iterations. Moreover, adaptive boundary treatments and non-uniform samplings can be 
easily implemented using interpolating methods. 

Compared with commonly used wavelet transforms, the interpolating wavelet 
10 transform possesses the following characteristics: 

1. The wavelet transform coefficients are generated by linear combination of 
signal samplings, instead of the commonly used convolution of wavelet transform, 
such as 

WjAx)/{x)dx (4) 

1 5 where {^.jt(x)=2^'^ vK^x-k). 

2. A parallel-computing algorithm can be easily constructed. The calculation and 
compression of coefficients are not coupled. For the halfband filter with length A^, the 
calculation of the wavelet coefficients, Wjj^, does not exceed 7VH-2 multiply/adds for 
each. 

20 3 . For a D-th order differentiable function, the wavelet coefficients decay rapidly. 

4. In a mini-max sense, threshold masking and quantization are nearly optimal 
for a wide variety of regularization algorithms. 

Theoretically, interpolating wavelets are closely related to the following 
wavelet types: 
25 Band-limit Shannon wavelets 

The ;rband-limited fimction, (x)=sm(x)/{x)C in Paley-Wienex space, constructs 
interpolating fimctions. Every band-limited fimction fL\R) can be reconstructed by 
the equation 

/(x)=E/(it)^iifi£:i*l 

30 k \ } 



where the related wavelet fimction (the Sinclet) is defined as (see Figure 1) 
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^ sin;r(2x-l)-sin;r(x-l/2) 

v'(^^= l^iTTTT) ^ 

Interpolating fundamental spline 

The fundamental polynomial spline of degree Z), rf(x), where D is an odd 
integer, has been shown by Schoenberg (1972) to be an interpolating wavelet (see 
Figure2). It is smooth with order 1, and its derivatives through order i)-l decay 
exponentially [34]. Thus 



(7) 



where y^(jc) is the B-spline of order D defined as 
Here U is the step function 

, [0,jc<() 

^<'' = {l..>() (9) 
and {aF (k)} is the sequence that satisfies the infinite summation condition 

Ya''(k)P''{n-k) = S(n) 

r (10) 

Deslauriers-Dubuc functional 

Let D be an odd integer, and D>0. There exist fiinctions, F^, such that if 
has aheady been defined at all binary rationals with denominator 2^, it can be extended 
by polynomial interpolation, to all binary rationals with denominator 2^', i.e. all 
points halfway between previously defined points [8, 11]. Specially, to define the 
fiinction at (k^V2)/2/ when it is already defined at all {kV} , fit a polynomial ^j,, to 

the data (A: 72', Fo(k72!) for k'€ {V [k-iD-\)/2] T\h¥(ih-l)/2]} . This polynomial is 

unique 



_ ,k + H2. ,k + l/2^ 



(11)- 
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This subdivision scheme defines a function that is uniformly continuous at the 
rationals and has a unique continuous extension. The function is a compactly 
supported interval polynomial and is regular; it is the auto-correlation function of the 
Daubechies wavelet of order £>+!. This function is at least as smooth as the 
corresponding Daubechies wavelets. 

Auto-correiation shell of orthonormal ^vavelets 

If ^ is an orthonormal scaling function, its auto-correlation 

^(x) = \htY h^' t)dt jg ^ interpolating wavelet (Figure3) [26]. Its smoothness, 

localization and two-scale relations derive from ^ . The auto-coirelation of Haar, 

Lamarie-Battle, Meyer, and Daubechies wavelets lead, respectively, to the 
interpolating Schauder, interpolating spline, C* interpolating, and Deslauriers-Dubuc 
wavelets. 

Lagrange half-band filters 

Ansari, Guillemot, and Kaiser [1] have used Lagrange symmetric halfband 
FIR filters to design the orthonormal wavelets that express the relation between the 
Lagrange interpolators and Daubechies wavelets [6]. Their filter corresponds to the 
Deslauriers-Dubuc wavelet of order Z)=7 (2A/-1), Af=4. The transfer function of the 
halfband symmetric filter h is given by 

2 (12) 

where Tis a trigonometric polynomial. Except for A(0)=l/2, at every even integer 
lattice /i(2/2)=0, n^^O, «62. The transfer function of the symmetric FIR filter h{n)-h{- 
n\ has the form 

// (z) = I / 2 + § h{2n - + z'"-' ) 

(13) 

The concept of an interpolating wavelet decomposition is similar to that of 
"algorithm a trous," the connection having been found by Shensa [27]. The self- 
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induced scaling conditions and interpolation condition are the most important 
characteristics of interpolating wavelets. According to the following equation 

Ax)=I^f{n)<f(x-n) (14) 
the signal approximation is exact on the discrete sampling points, which does not hold 
5 in general for commonly used non-interpolating wavelets. 



LAGRANGE WAVELETS 

Halfband Lagrange Wavelets 

The halfband filter is defined as whose even samples of the impulse response 
10 are constrained such as /i(0)=l/2 and A(2«)=0 for n=±l, ±2,., . A special case of 

symmetric halfband filters can be obtained by choosing the filter coefficients- 
according to the Lagrange interpolation formula. The filter coefficients are then given 
by 

15 ^^'^'^ ■ " (M - n)\{M\ n - l)!(2n - 1) (15) 

These filters have the property of maximal flatness. They possess a balance between 
the degree of flatness at zero frequency and flatness at the Nyquist frequency (half 
sampling). 

20 These half-band filters can be utilized to generate the interpolating wavelet 

decomposition, which can be regarded as a class of auto-correlated shell ojthogonal 
wavelets such as the Daubechies wavelets [6], The interpolating wavelet transform 
can also be generated by different Lagrange polynomials, such as [26] according to 

The predicted interpolation can be expressed as 

= J /'j,_|(0)[5^(/ + 2/1 - 1) + 5/1- 2/1 + 1)]' 

(17) 

30 ^2/^1. 

where T is a projection and the are the yth layer low-pass coefficients. This 
projection relation is equivalent to the subband filter response of 
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K2rtA)^P^.,iO) (18) 
The above-mentioned interpolating wavelets can be regarded as an extension 
of the fundamental Deslauriers-Dubuc interactive sub-division scheme (factorized as 
Af=2, while the order of Lagrange polynomial is D=2M-l=3) (Figure 6(a)). 

It is easy to verify that an increase of the Lagrange polynomial order will 
introduce higher regularity in the interpolating fimctionals (Figure 7(a)). When 
jD->-h», the interpolating functional becomes the -band-limited Sine function and its 
domain of definition is the real line. The subband filters generated by Lagrange 
interpolating functional satisfy 

(1) Interpolation: A(d?)+/i(<aH-;^=l 

(2) Symmetry: /i(fi;)=/z(-^y) 

(3) Vanishing Moments: ^xf'^x)dx=Sp 

Donoho outlines a basic subband extension for perfect reconstruction. He 
defines the wavelet function as 

Kx)=A2xA) (19) 
The biorthogonal subband filters can be expressed as 

Mfl>) = l, g(^u) = e-"". g(fi)) = e-'"/i(a> + ^r) (20) 

However, the Donoho interpolating wavelets have some drawbacks, because the low- 
pass coefficients are generated by a sampling operation only, as the decomposition 
layer increases, the correlation between low-pass coefficients becomes weaker. The 
interpolating (prediction) error (high-pass coefficients) strongly increases, which 
destroys the compact representation of the signal. Additionally, it does not lead to a 
Riesz basis for L\R) space. 

Swelden has provided, by far, the most efficient and robust scheme [32] for 
constructing biorthogonal wavelet filters. His approach is to generate high-order 
interpolating Lagrange wavelets with increased regularity. As Figure 4 shows, is 
the interpolating prediction process, and P, the so-called the updating filter, makes the 
down-sampling low-pass coefficients smoother. If we choose Pq to be the same -as P,, 
then the new interpolating subband filters can be depicted as 
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(21) 

^i(^)*e""'-/rM/>(2<w) 
?i(<w) = ?(<y) 

The newly developed filters A„ g,, f^^ , and also construct the biprthogonal 

dual pair for perfect reconstruction. Examples of generated biorthogonal lifting 
5 wavelets with different regularity are shown in Figure 6 and Figure 7. 

B-Spline Lagrange Wavelets 

Lagrange polynomials are natural interpolating expressions. Utilizing a 
different expression for the Lagrange polynomials, we can construct other types of 
interpolating wavelets. 

10 We define a class of symmetric Lagrange interpolating fimctional shells as 

^ X-/ 



^A/W- n — r (22) 



It is easy to verify that this Lagrange shell also satisfies the interpolating condition on 
1 5 discrete integer points, since 



A: = 0 

otherwise (23) 



However, simply defining the filter response as 
20 h{kyP{kl2)l2, M (24) 

will lead to non-stable interpolating wavelets, as shown in Figure 9. 

Utilizing a smooth window, which vanishes at the zeros of the Lagrange 

polynomial, will lead to more regular interpolating wavelets and equivalent subband 

filters (as in Figures 10 and 1 1). If we select a well-defined B-spline function as the 
25 weight window, then the scaling function (mother wavelet) can be defined as an 

interpolating B-Spline Lagrange fimctional (BSLF) 

^^W = -^^m(^) ^ 

P^xln) fr ^^^^ 

30 where N is the B-spline order, and tj is the scaling factor to control the window width. 

To ensure coincidence of the zeroes of the B-spline and the Lagrange polynomial, we 
set 



wo 99/46731 



PCT/US99/05426 



74 

2M=7x(AH-l) (26) 
To preserve the interpolation condition, the B-spline envelope factor M must be odd 
number. It is easy to show that when the B-spline order is N^Ak^l^ 7j canl)e any odd 
integer (2*+l). If i\^is an even integer, then 7 can only be 2. When A^=^4^-l_, we cannot 
construct an interpolating shell according to the above definition. 

From the interpolation and self-induced scaling of the interpolating wavelets, 
it is easy to establish that 

Kk)^(ffJim)ll, /c=-2M+l, 2M'\ {11) 
Gaussian-Lagrange DAF Wavelets 

We can also select a distributed approximating functional-Gaussian Lagrange 
DAF (GLDAF) as our basic scaling function to construct interpolating wavelets. 
These are 

= ^^.W n (28) 
where WJx) is a window function which is selected to be a Gaussian, 

Ki^>e-''''-\ (29) 

because it satisfies the minimxim frame bound condition in quantum physics. Here a 
is a window width parameter, and Pjj(x) is the Lagrange interpolation kernel The 
DAF scaling function has been successfully introduced as the basis for an efficient 
and powerful grid method for quantum dynamical propagations [40]. Using the Ufling 
scheme [32], a wavelet basis is generated. The Gaussian window in our DAF- 
wavelets efficiently smooths out the Gibbs oscillations, which plague most 
conventional wavelet bases. The following equation shows the connection between 
the B-spline function and the Gaussian window [34]: 

for large N. As in Figure 12, if we choose the window width to be 
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. (31) 

£r=7V(// + l)/I2. 

the Gaussian Lagrange wavelets generated by the lifting scheme will -be. much like the 
B-spline Lagrange wavelets. Usually, the Gaussian Lagrange DAF based wavelets are 
smoother and decay more rapidly than B-spline Lagrange wavelets. If we select more 
sophisticated window shapes, the Lagrange wavelets can be generalized further. We 
shall call these extensions Bell-windowed Lagrange wavelets. 



VISUAL GROUP NORMALIZATION 

It is well known that the mathematical theory of wavelet transforms and 
associated multiresolution analyses has appHcations in signal processing and. 
engineering problems, where appropriate subband filters are the central entities. The 
goal of wavelet signal filtering is to preserve meaningful signal components, while 
efficiently reducing noise components. To this end, we use known magnitude 
normalization techniques [28, 29] for the magnitudes of filter coefficients to develop a 
new perceptual normalization to account for the human vision response. 

From a signal processing point of view, wavelet coefficients can be regarded 
as the results of the signal passing through equivalent decomposition filters (EDF). 
The responses of the EDF LC^J^o)) are the combination of several recurrent subband 
filters at different stages. As shown in Figure 6., the EDF amplitudes of different sub- 
blocks are different. Thus the magnitude of the decomposition coefficients in each of 
the sub-blocks will not exactly reproduce the true strength of the signal components. 
Stated differently, various EDFs are incompatible with each other in the wavelet 
transform. To adjust the magnitude of the response in each block, the decomposition 
coefficients are re-scaled with respect to a common magnitude standard. Thus EDF 
coefficients, Cj JJc), on layer j and block m should be multiplied by a magnitude 
scaling factor, to obtain an adjusted magnitude representation [28]. This factor 
can be chosen as the reciprocal of the maximum magnitude of the firequency response 
of the equivalent filter on node Q\m) 
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This idea was recently extended to Group . Normalization (GN) of wavelet packets for 
signal processing [29]. 

An image can be regarded as the result of a real object processed by a human 
visual system. The latter has essentially many subband filters. The responses of these 
human filters to various frequency distributions are not at all imiform. therefore, an 
appropriate normalization of the wavelet coefficients is necessary. Actually, the 
human visual system is adaptive and has variable lens and focuses for different visual 
environments. Using a just-noticeable distortion profile, we can efficiently remove the 
visual redundancy from decomposition coefficients and normalize them with respect 
to a standard of perception importance, A practical, simple model for perception 
efficiency has been presented by Watson, et al. [5] for data compression. This model 
is adapted here to construct the "perceptual lossless" response magnitude Yj^ for 
normalizing according to the visual response function [39], 



where a defines the minimum threshold, kisa constant, R is the Display Visual 
Resolution (DVR),yo is the spatial frequency, and d„ is the directional response 
factor. Together with the magnitude normalized factor Aj^, this leads to the perceptual 
lossless quantization matrix 



This treatment provides a simple, human-vision-based threshold technique 
[39] for the restoration of the most important perceptual information in an image. For 
gray-scale image processing, the luminance (magnitude) of the image pixels is the 
principal concern. We refer to the combination of the above mentioned two 
normalizations as the Visual Group Normalization (VGN) of wavelet coefficients. 

MASKING TECHNIQUE 

Masking is essential to many signal-processing algorithms. Appropriate 
masking will result in reduction of noise and undesired components. Certainly, it is 
very easy to set up masking if the spectral distribution of a signal and its noise is 
known. However, in most cases, such prior knowledge is not available. Statistical 




(33) 




(34) 
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properties of the signal and its noise are assumed so that the noise is taken to be 
relatively more random than the signal in each subband. Hard logic masking and soft 
logic masking techniques are discussed in the following two subsections. 
Hard Logic Masking 

5 The Visual Group Normalization method provides an efficient approach for 

re-normalizing the wavelet decomposition coefficients so that various subband filters 
have appropriate perceptual impulse responses. However, this algorithm alone does 
not yield the best SNR in real signal processing. Essentially, various noise and/or 
interference components can be embedded in different nodes of the subband 

10 decomposition tree. To achieve SNR-improved reconstruction of the signal and/or 

image, a filtering process is needed to reduce the noise and preserve the main signal 
information. Noise due to random processes has a comparatively wide-band 
distribution over the decomposition tree, whereas mechanical noise can have a 
narrow-band distribution over a few specific subband components. Therefore, time- 

15 varying masking techniques are utilized to reduce noise. We discuss a few usefiil 

masking methods in the rest of this subsection. 

Single Dead-Zone Threshold Masking 
A single zone threshold masking is the simplest masking method. With a 
given decomposition tree, a constant threshold r is selected for our magnitude 

20 nonnalized wavelet decomposition coefficients NCj^{k). That is, if the absolute value 

of NCj„{k) is greater than the threshold r, the original decomposition coefficient will 
be kept; otherwise it will be set to zero. That is 

^-<''-|0 \NC,Jk)\<r (35) 

25 

This approach is similar to the Pass-Band Selection technique of Ref. [43], 
used in a FFT firamework. However, in the present approach, the decomposition 
coefficients are re-scaled using the visual group normalization. Therefore, even with a 
single zone masking, it is expected that for a given noisy signal the present wavelet 
30 analysis will achieve a better SNR than that of a single-band FFT method. 

Adaptive Node MeanA^ariance Threshold Masking 
In practice, we hope the threshold r can be adaptively adjusted to the strength 
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of the noisy environment. Thus the threshold r should be set higher to suppress a 
noisier signal, and in general, r should vary as a function of the statistical properties 
of the wavelet decomposition coefficients, the simplest and most important of which 
are the mean and second variance. These are incorporated in the present work. 

We define the mean and second variance of the magnitude of the nomialized 
coefficients on node as 



1 ^* 



(36) 



10 and 



(37) 



where A^==2'W and N is the total length of a filter. A masking is then set according to 
15 the following analysis: 

(1) Introduce a factor \c\<\^ aeR. 

(2) Set the corresponding wavelet transform coefficients, Cj JJc), to zero, if the 
inequality, \NCjJ]c)\<\7jj^'^aajJ{ holds. This implies that the magnitude of the 
normalized coefficients NCj JK) is less than the mean, rjj^, within the statistical 

20 deviation of oco-^j and hence that NCj „^(Jc) as a noise component. 

(3) Retain the corresponding wavelet transform coefficients, Cj JJc)^ for 
reconstruction, satisfying the inequality, |7VC^,„(A:)|>|;;^ ,„+arc5„I, This implies that the 
magnitude oiNCj JJc) is greater than the mean, 7^^, within the statistical deviation of 
aaj^. We consider such a NCj„(k) as a target component. The rules (2) and (3) are 

25 summarized as 



<K^c.a,\ (38) 



Adaptive Whole Tree Threshold Masking 
30 For certain applications, it is possible that all NCj JK) on a particular node 

{j,m) have essentially the same values. The aforementioned adaptive node 
mean/variance threshold masking technique becomes invalid in such a case. We use 
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an adaptive whole-tree threshold masking method for this situation. The basic 
procedure is very similar to Method 2, except that the mean and second variance are 
calculated for the whole tree T, according to 

(39) 

and 

^=[7F2IZK.W-7f J {i,m)^T. (40) 

The corresponding reconstruction coefficients are selected by the rules 

r l^J.W |A^C,„(A)l>|7^aal 



1 5 Constant False Alarm Masking 

In certain applications, such as radar signals generated fi*om a given 
environment, it is useful to select an alarm threshold, r, based on the mean value of 
multiple measurements of the background signal. This approach is similar to a 
background-contrasted signal processing in which only the differences of the signal's 
20 optimal tree decomposition coefficients, NCj JJi), and n Cy\(*) » the background 

decomposition coefficients of the same tree structure, are used for backgrojmd-" 
contrasted signal reconstruction. 

Softer Logic Masking 

The various maskings discussed above can be regarded as hard logic masking, 
25 which are similar to a bias-estimated dead-zone limiter. Jain [16] has shown that a 

non-linear dead-zone limiter can improve the SNR for weak signal detection 

r7{;') = sen{>')(M-^)'.-l^>?^l (42) 



30 



where <Jis a threshold value. Donoho has shown that the >^1 case of the above 
expression is a nearly optimal estimator for adaptive NMR data smoothing and de- 
noising [9]. Independently, two of the present authors (Shi and Bao) in a previus work ' 
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[28] have utilized hard logic masking to extract a target from formidable background 
noise efficiently. 

The various threshold cutoffs of multiband expansion coefficients in hard 
logic masking methods are very similar to the cutoff of a FFT expansion. Thus, Gibbs 
oscillations associated with FFTs will also occur in the wavelet transform using a hard 
logic masking. Although hard logic masking methods with appropriate threshold 
values do not seriously change the magnitude of a signal after reconstruction, they can 
cause considerable edge distortions in a signal due to the interference of additional 
high frequency components induced by the cutoff The higher the threshold value, the 
larger the Gibbs oscillation will be. Since image edges are especially important in 
visual perception, hard logic masking can only be used for weak noise signaL(or 
image) processing, such as electrocardiogram (ECG) signal filtering, where relatively 
small threshold values are required. In this paper, we use a Soft Logic Masking 
(SLM) method. In our SLM approach, a smooth transition band near each masking 
threshold is introduced so that any decomposition coefficients, which are smaller than 
the threshold value will be reduced gradually to zero, rather than being exactly set to 
zero. This treatment efficiently suppresses possible Gibbs oscillations and preserves 
image edges, and consequently improves the resolution of the reconstructed image. 
The SLM method can be expressed as 

C,^(*) = sgn(c,^(A))^C,^(/:)l-<j);x5{iVC^ ^^3^ 

where Q. m(Jc) are the decomposition coefficients to be retained in the reconstruction 



and quantity NQ, m{k) is defined as 



m^^{\NC,Sk)\} (44) 



The softer logic mapping, iS:[0,l]->[0,l], is a non-linear, monotomcally 
increasing sigmoid functional. A comparison of the hard and softer logic masking 
functional is depicted in Figure 13. 

In 2D image processing, it is often important to preserve the image gradient 
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along the xv-direction. For this puipose, we modify the aforementioned softer logic 
functional to 



where ^is a normalized adaptive threshold. For an unknown noise level, an useful 
approximation to ^is given by 



where u is a scaling factor conveniently chosen as a =1/1 .349. The quantity Yupptr is 
an upper frame boundary of the wavelet transform, i.e. the upper boundary singular 
value of the wavelet decomposition matrix. Using arguments similar to those given by 
Donoho [9], one can show that the above Softer Logic Masking reconstruction is a 
nearly optimal approximation in the minimax error sense. 

EXPERIMENTAL RESULTS 

To test our new approaches, standard benchmark 512x5 12 Y-component images 
are employed. Generally, the possible noise sources include photoelectric exchange, 
photo spots, the error of image communication, etc. The noise causes the visual 
perception to generate speckles, blips, ripples, bumps, ringings and aliasing. The noise 
distortion not only affects the visual quality of the images, but also degrades the 
efficiency of data compression and coding. De-noising and smoothing are extremely 
important for image processing. 

The traditional image processing techniques can be classified as two kinds: linear 
or non-linear. The principle methods of linear processing are local averaging, low-pass 
filtering, band-limit filtering or multi-frame averaging. Local averaging and low-pass 
filtering only preserve the low band frequency components of the image signal. The 
original pixel strength is substituted by an average of its neighboring pixels (within a 
square window). The mean error may be improved but the averaging process will blur 
the silhouette and finer details of the image. Band-limited filters are utilized to remove 
the regularly appearing dot matrix, texture and skew lines. They are useless for noise 




(45) 



(46) 
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whose correlation is weaker. Multi-frame averaging requires that the images be still, and 
the noise distribution stationary. These conditions are violated for motion picture images 
or for a space (time)-varying noisy background. 

The traditional image quality is characterized by a mean square error {MSE), 
5 which possesses the advantage of a simple mathematical structure. For a discrete signal 

{s(n)} and its approximation {s(n)}, «=0, . . ., A^, the MSE can be defined to be 

N,^ (47) 

10 However, the MSE based evaluation standard, (such as 

P5'A«=log[(255'255)/MiSE]), can not exactly evaluate the image quality if one neglects 
the effect of human perception. The minimum MSE rule will cause strong undulations 
of the image level and destroy the smooth transition information around the pixels. 
Commonly used regularization methods may degrade the image edges and result in visual 

15 blur 

Generally, unsatisfactory traditional image processing is always defined on the 
whole space (time) region, which does not localize the space (time)-frequency details of 
the signal. New theoretical research shows that non-Guassian and non-stationary 
characteristics are important components in human visual response. Human visual 

20 perception is more sensitive to image edges which consist of sharp-changes of the 

neighboring gray scale because it is essentially adaptive and has variable lens and focuses 
for different visual environments. To protect edge information as well as remove noise, 
modem image processing techniques are predominantly based on non-linear methods. 
Before the smoothing process, the image edges, as well as perceptually sensitive texture 

25 must be detected. The commonly used non-linear filtering approaches include median 

filtering, and weighting average, etc. Median filtering uses the median value within the 
window instead of the original value of the pixel. This method causes less degradation 
for slanted fimctions or square fimctions, but suppresses the signal impulses, which are 
shorter than half of the window length. This will degrade the image quality. The most 

30 serious shortcomings of weighting average method are that the weighting-window is not 

adaptive, and large-scale, complicated calculations are required to generate pixel values. 
If the window is made wider, more details will be removed. 
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More efficient human-vision-system-based image processing techniques possess 
the -advantages of 1) large range de-correlation for convenience of compression and 
filtering; 2) high perceptual sensitivity and robustness; 3) filtering accordmg to human 
visual response. It therefore can enhance the most important visual information, such as 
edges, while suppressing the large scale of flat regions and background. In addition 4) 
it can be carried out with real-time processing. 

The space (time)-scale logarithmic response characteristic of the wavelet 
transform is similar to the HVS response. Visual perception is sensitive to narrow band 
low-pass components, and is insensitive to wide band high fi'equency components. 
Moreover, fi-om research in neurophysiology and psychophysical studies,'the direction- 
selective cortex filtering is very much like a 2D-wavelet decomposition. .The high-pass 
coefficients of the wavelet transform can be regarded as the visible difference predictor 
(VDP). 

Utilizing the modified wavelet analysis-Visual Group-Normalized Wavelet 
Transform ( VGN-WT) presented in this paper, we can correct the drawback that the raw 
magnitudes of the transform coefficients do not exactly yield the perceptual strength of 
digital images. By use of the soft logic masking technique, the non-Unear filtering 
provides edge-preservation for images, which removes the haziness encountered with 
commonly used filtering techniques. 

The first test result is for the so-called "Lena" image, which possesses clear sharp 
edges, strong contrast and brightness. The second picture tested is ' 'Barbara". The high 
texture components and consequently high fi'equency edges in the Barbara-image create 
considerable difficulties for commonly used filtering techniques, A simple low-pass filter 
will smooth out the noise but will also degrade the image resolution, while a simple high- 
pass filter can enhance the texture edges but will also cause additional distortion. 

We choose 2D half-band Lagrange wavelets as the testing analysis tools for image 
processing. The four 2D wavelets are shown in Fig. 14. 

As shown in Fig. 15(a) and Fig. 16(a) respectively, adding Gaussian random noise 
degrades the original Lena and Barbara images. The nledian filtering -(with a 3x3 
window) result of Lena is shown in Fig. 1 5(b). If is edge-blurred with low visual quality; 
the speckled noise has been changed into bumps. This phenomenon is even more serious 
for the Barbara image, which has many textures and edges. These features always create 
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severe difficulties for image restoration. As shown in Fig. 15(c) and Fig. 16(c), it is 
evident that our perceptual DAF wavelet technique yields better contrast and edge- 
preservation results and provides high quality visual restoration performance. 

5 CONCLUSION 

This paper discusses the design of interpolating wavelets based on Lagrange 
functions, as well as their application in image processing. A class of biorthogonal 
Lagrange interpolating wavelets is studied vnth regard to its application in signal 
processing (especially for digital images). The most attractive property of the 
1 0 interpolating wavelets is that the wavelet multiresolution analysis is realized by discrete 

sampling. Thus pre- and post-conditioning processings are not needed for an^ accurate 
wavelet analysis. The wavelet coefficients are obtained firom linear combinations of 
sample values rather than from integrals, which implies the possibility of using parallel 
computation techniques. 

15 Theoretically, our approach is closely related to the finite element technique for 

the numerical solution of partial differential equations, the subdivision scheme for 
interpolation approximations, multi-grid methods and surface fitting techniques. In this 
paper, we generalize the definition of interpolating Lagrange wavelets and produce three 
different biorthogonal interpolating Lagrange wavelets, namely Halfband Lagrange 

20 wavelets, B-spline Lagrange wavelets and Gaussian-Lagrange DAF wavelets. 

Halfband Lagrange wavelets can be regarded as an extension of-Dubuc 
interpolating functionals, auto-correlation shell wavelet analysis and halfband fiUers. B- 
spline Lagrange Wavelets are generated by B-spline windowing of a Lagrange functional, 
and lead to increased smoothness and localization compared to the basic Lagrange 

25 wavelets. 

Lagrange Distributed Approximating Functionals (LDAF) can be regarded as 
scaling functions (wavelet-DAFs) and associated DAF-wavelets can be generated in a 
number of ways [41], Both DAFs and DAF-wavelets are smoothly decay in both time 
and frequency representations. The present work extends the DAF approach to signal and 

30 image processing by constructing new biorthogonal DAF-wavelets and associated D AF- 

filters using a lifting scheme [32]. . 

In the first part of our image processing application, we combine two important - 
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techniques, the coefficient normalization method and softer logic visual masking based 
on Human Vision Systems (HVS). The resulting combined technique is called Visual 
Group Normalization (VGN) processing [31]. The concept of Visual Loss-less 
Quantization (VLQ) presented in [39] can lead to a potential breakthrough compared to 
the traditional Shannon rate-distortion theory in information processing. 

We also employ a modified version of Donoho's Soft Threshold method for image 
restoration, termed the Softer Logic Perceptual Masking (SLM) technique, for dealing 
with extremely noisy backgrounds. This technique better preserves the important visual 
edges and contrast transition portions of an image than the traditional Donoho method 
and is readily adaptable to human vision. 

Computational results show that our Lagrange wavelet based VGN processing is 
extremely efficient and robust for digital image blind restoration and yields the best 
performance of which we are aware, when applied to standard Lena and Barbara images. 
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IMAGE ENHANCEMENT NORMALIZATION 
INTRODUCTION 

Medical image, such computed tomography (CT), magnetic resonance image 
(MRI), X-ray mammogram, ultrasound and angiography, is one of major methods for 
field diagnosis. In particular, X-ray mammogram is widely recognized as being the only 
effective method for the early detection of breast cancer. Major advances in screen/film 
mammograms have been made over the past decade, which result in significant 
improvements in image resolution and film contrast without much increase in X-ray dose. 
In fact, mammogram films have the highest resolution in comparison to various other 
screen/film techniques. However, many breast cancers cannot be detectedjust based on 
mammogram images because of poor visualization quality of the image.. This is due to 
the minor difference in X-ray attenuation between normal glandular tissues and 
malignant disease, which leads to the low-contrast feature of the image. As a result, early 
detection of small tumors, particularly for younger women who have denser breast tissue, 
is still extremely difficult. 

Mammogram image processing has drawn a great deal of attention in the past 
decades [1-9]. Most work focuses on noise reduction and feature enhancement. The 
statistical noise level is relatively low for images obtained by the modem data acquisition 
techniques. Therefore feature enhancement is more essential for mammogram quality 
improvement. Since an image noise reduction algorithm was reported earlier [10], we 
shall focus on image enhancement in this work. In our opinion, there are two basic ideas 
for manunogram feature improvement. One type of methods is to change inmge spectrum 
distribution so as to increase image edge densities. As a result, the image appears sharper. 
The other approach is the so-called image windowing, in which linear or nonlinear 
grayscale mappings are defined so as to increase/compress image gradient density 
according to device. 

In an early work, one of present authors realized that the firequency responses of 
wavelet transform subband filters are not xmifomi. A magnitude normalization algorithm 
was proposed to account for this uniform feature [11, 12] . This idea was later extended 
to group normalization for the wavelet packet transform [13] and visual group 
normalization for still image noise reduction [10] by present authors. In this work we 
fiirther extend this idea to achieve image enhancement. 
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Our first approach is based on a rearrangement of image spectrum distribution by 
edge enhancement normalization of wavelet coefficients. Our second technique utilized 
a multiscale functional to obtained device adapted visual group normalization of wavelet 
coefficients. Numerical experiments indicate that our normalization approach provides 
excellent enhancement for low quality mammogram image in combination with the use 
of the biorthogonal interpolating wavelets [10] generated by Gaussian Lagrange 
distributed approximating fimctionals [14]. 

EDGE ENHANCEMENT NORMALIZATION 

Mallat and Zhong realized that Wavelet multiresolution analysis provides a 
natural characterization for multiscale image edges, and those manipulations can be 
easily achieved by various differentiations [15]. Their idea was extended by Laine et al 
[7] to develop directional edge parameters based on subspace energy measurement. An 
enhancement scheme based on complex Daubechies wavelets was proposed by Gagnon 
et al. [9]. These authors made use of the difference between real part and imaginary part 
of the wavelet coefficients. One way or another, distorted wavelet transforms are 
designed to achieve desired edge enhancement. 

Our starting point is magnitude normalized or visual group normalized wavelet 
subband coefficients NCj„,{k) [10, 12]. We define an enhancement functional Ej„ 

(1) 

where A is the Laplacian and -l<cry,„, Coefficients aj„^y can be easily chosen 

so that desired image features are emphasized. In particular it enables us to emphasize 
an image edge of selected grain size. We note that a slightly modification of ^ „ and fij ^ 
can result in orientation selected image enhancement. A detailed discussion of this matter 
will be presented elsewhere. An overall re-normalization is conducted after image 
reconstruction to preserve the energy of the original image. We call this procedure 
enhancement normalization, 

DEVICE ADAPTED VISUAL GROUP NORMALIZATION 

Contrast stretching is old but quite efficient method for feature selective image 
display. Nonlinear stretching has been used by many authors [3, 7, and 16]. Lu and 
coworkers [16] has recently designed hyperbolic function g^(A:)=[tenh(flA:- 
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6)-i-tanh(6)]/[taiih(a-6)+tanh(6)] for wavelet multiscale gradient transformation. Their 
method works well for moon images. The basic idea is to use gradient operators to shape 
a fat image data so that desired portion of the image is projected into a screen window. 
However, in most approaches, the perceptual response of human visual system is not 
appropriately accounted. The human visual system is adaptive and has variable lens and 
focuses for different visual environments. We propose a himian visual response corrected 
grayscale gradient mapping technique for selected contrast enhancement. 

The human visual system is adaptive and has variable lens and focuses for 
different visual environments. Using a just-noticeable distortion profile, one can 
efficiently remove the visual redundancy from the decomposition coefficients [17] and 
normalize them with respect to the importance of perception. A practical simple model 
for perception efficiency has been presented to construct the "perceptual lossless" 
response magnitude Yj „ for normalizing according to visual response, 



where ^ is a constant, R is the Display Visual Resolution (DVR), is the spatial 
frequency, and d„ is the directional response factor. A perceptual lossless quantization 
matrix Qj „ is [10] 



where Aj^ is a magnitude normalized factor. This treatment provides a simple-human- 
vision-based threshold technique for the restoration of the most important perceptual 
information in an image. For grayscale image contrast stretching, we first appropriately 
normalize the decomposition coefficients according to the length scale, I, of the display 
device [16] so that they fall in interval of [0,1] of the device frame 




(2) 




(3) 




(4) 



NCj^.^rMNCjJ 



(5) 



where constant and function ^„ is appropriately chosen so that desired portion of the 
grayscale gradient is stretched or compressed. For example, function 
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can been tuned to stretch any portion the of grayscale gradient. This procedure shall be 
called device adapted visual group normalization. 

EXPERIMENTAL RESULT 

5 To test our new approaches, low-contrast and low quality breast mammogram 

images are employed. Mammograms are complex in appearance and signs of early 
disease are often small or subtle. Digital mammogram image enhancement is particularly 
important for aiding radiologists and for the development of automatically detecting 
expert system. A typical low quahty front-view image is shown in Figure 1(a). The 

1 0 original image is coded at 5 1 2x5 1 2 pixel size with 2 bytes/pixel and 1 2 bitsof gray scale. 

We have conducted edge enhancement normalization and device adapted visual group 
normalization. As shown in Figure 1(b), there is a significant improvement in both edge 
representation and image contrast. In particular, the domain and internal structure of 
high-density cancer tissues are clear displayed. Figure 2(a) is an original 1 024x 1 024 side- 

1 5 view breast image which has been digitized to 200 micron pixel edge with 8 bits of gray 

scale. Enhanced image result is depicted in Figure 2(b). In this case we obtain a similar 
result in the previous one. 
CONCLUSION 

Edge enhancement normalization (EEN) and device adapted visual group 
20 normalization (D AVGN) are proposed for image enhancement without prior knowledge 

of the spatial distribution of the image. Our algorithm is a natural extension of earlier 
normalization techniques for image processing. Biorthogonal interpolating distributed 
approximating functional wavelets are used for our data representation. Excellent 
experimental performance is found for digital mammogram image enhancement. 
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VARYING WEIGHT TRIMMED MEAN FILTER FOR THE RESTORATION OF 

IMPULSE CORRUPTED IMAGES 

INTRODUCTION 

Images are often corrupted by impulse noise due to a noisy sensor or channel 
transmission errors. Impulse interference may be broadly defined as corruption which is 
random, sparse, and of high or low amplitude relative to local signal values. Impulse 
noise seriously affects the performance of various signal processing techniques, e.g., edge 
detection, data compression, and pattern recognition. One of the tasks of image 
processing is to restore a high quality image fi-om a corrupted one for use in subsequent 
processing. The goal of image filtering is to suppress the noise while preserving the 
integrity of significant visual information such as textures, edges and details. Nonlinear 
techniques have been found to provide more satisfactory results than linear methods. One 
of the most popular nonlinear filters is the median filter [1-2], which is well-known to 
have the required properties of suppressing the impulse noise and preserving the edges. 
However, it is also true that the median filter is not optimal. It suppresses the true signal 
as well as noise in many applications. Many modifications and generalizations of the 
median filter have been proposed [3-8] for improving its performance. 

In reference [8], we presented a generalized trimmed mean filter (GTMF),which 
is essentially a generalization to the alpha-trimmed mean filter (a-TMF) [5-6], In GTMF, 
a ' 'median basket" is employed to select a predetermined number of pixels on both sides 
of the median value to the sorted pixels of the moving window. The luminance values of 
the selected pixels and the value of the center pixel in the window are then weighted and 
averaged to give the filtering output. As mentioned in [8], it is importaiiMo have the 
center pixel participate in the averaging operation when removing additive noise, but one 
usually sets the weight of the center pixel to zero when filtering impulse-noise-corrupted 
images. 

Although the GTMF outperforms many well known methods in removing highly 
impulse noise corrupted images, it can be finiher modified to improve the filtering 
performance. In GTMF, the averaging weights are predetermined and fixed throughout 
the filtering procedure. In this paper, a varying weight fiihction is designed and applied 
to GTMF. We give it a new name, varying weight trinuned mean filter (VWTMF).In 
VWTMF, the argument of the weight function is the absolute difference between the 
luminance values in the median basket and the median value. Because we only' 



wo 99/46731 



PCT/US99/05426 



95 

concentrate on filtering impulse-noise-corrupted images in this paper, the weight of the 
center pixel in the moving window is always assumed to be zero. For most effectively 
removing impulse noise, we combine the VWTMF with a switching scheme [9] as a 
impulse detector and an iterative procedure [8] is employed to improve the filtering 
performance. 

Varying Weight Trimmed Mean Filter 

The pixels {/y, Z^, . . . , 4.|, /„, ...,/„} in the moving window associated 
with a pixel 7^, have been sorted in an ascending (or descending) order in the same way 
as in the conventional median filtering technique, with /„, being the median value. The 
key generalization to the median filter which is introduced in the alpha-trimmed mean 
filter (a-TMF) [5,6] is to design a median basket in which to collect a giyen number of 
pixels above and below the median pixel. The values of these pixels are then averaged 
to give the filtering output, A^, as an adjusted replacementvalue to according to 

where I=La«J, with 0^a<0.5. It is evident that a single-entry median basket (i=0) 
a-TMF is equivalent to the median filter and a «-entry median basket (L=^\0.5]) is 
equivalent to the simple moving average filter. The generalized trimmed mean filter 
(GTMF) [8] uses a median basket in the same way as is does in the a-TMF to collect the 
pixels. The luminance values in themedian basket and the center pixel in the window 
are thenweighted and averaged to give the GTMF output: 

Gc = , 

E til, (2) 

where is the GTMF output, ana and the Wj's are the weights. It is interesting to see 

that when >v,.=0 and all w-s are equal to each other (nonzero), the GTMF becomes the 

a-TMF. When all weights except are equal to zero, it becomes the standard median 

filter. In the GTMF, the values of the weights are predetermined before filtering and are 

fixed during the filtering procedure. However, it is possible to further improve the GTMF 

by varying the weights according to absolute difference between the luminance values 

in the median basket and the median value. For the removal of impulse noise, we set 

>v^=0 and modify the GTMF by varying the weights, so the varying weight trimmed mean- 
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filter (VWTMF) is given by 




(3) 



where, Xj is a value in the range of [0,1] defined by 



(4) 



B 



with B being the maximum pixel value of a given type of image (e.g., B=255 for a 8 bit 
gray-scale image),The weight w{x) in Eq. (3) is a decreasing function in the range [0,1] 
and is taken to be 



The above weight function for A=2 is displayed in Figure 1 . Notice that w(0) = 
1 and w( 1 )=0, w{Xj^)= 1 is always the largest weight and the greater the difference between 
the pixel value and the median value, the smaller will be the weight. We anticipate that 
the VWTMF will outperform both the median filter and the a-TMF in suppressing 
impulse noise, while preserving the image edges. As is well known, the median value has 
the least probability to be impulse noise corrupted because the impulses are typically 
presented near the two ends of the sorted pixels. However, the median value may not be 
optimal because it may differ significantly from the noise-free value. The a-TMF will not 
perform better than the median filter when treating impulse noise corrupted images in the 
absence of any other technique (such as the switching scheme) because the corrupted 
pixels may be also selected to the median basket for the averaging operation. In contrast, 
the VWTMF which uses a weighted averaging operation can alleviate the shortcomings 
of both filters. The weight of the median value is the large stand the weights of other 
luminance values in the median basket vary according to their differences from the 
median value. If a impulse corrupted pixel happens to be selected for inclusion in the 
median basket, its contribution to the average will be small because Xj is large. In general, 
the weight function can assist the VWTMF in eliminating the impulse noise while 
providing a well adjusted replacement value for the center pixel 4. 




(5) 
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It must be noted that although the present implementation o fthe VWTMF 
employs Eq. (5) as the weight function, other monotonically decaying functions in the 
range of [0,1] may also be selected. 

THE SWITCHING SCHEME BASED ITERATIVE METHOD 

Many algorithms have been proposed to detect and replace impulse noise 
corrupted pixels of a image [9-12]. In this paper, a switching scheme similar to that used 
in reference [9], based on the VWTMF, is employed to detect impulse noise and recover 
the noise-free pixels. The filtering output 7^. is generated according to the following 
algorithm 



where /*'^ is the initial input value and is given in Eq. (3). A threshold T is chosen 
to characterize the absolute difference between and V^, If the difference is larger 
the threshold, it implies that the pixel differs significantly from its neighbors. It is 



difference is smaller than the threshold, it implies that the original is similar to its 
statistical neighbors, and we identify it as noise-free pixel, and it therefore retains its 
original value. 

Iteration of the above scheme will further improve the filtering performance, 
especially for images that are highly corrupted by impulse noise. The iteration procedure 
can be expressed as 



where 4(0 is the system output at time t with 7^(0)= J^^ , V^t) is the VWTMF filtering 
output given by 




(6) 



therefore identified as an impulse noise corrupted pixel, and is then replaced by V^. If the 



m = 



m \i^^-Vcit)\<T 



(7) 



E w{xj\t-l)I^{t-i) 



(8) 



Vcit) = 
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Note that it is important for the iterative procedure always to compare V^(i) with 
the initial input J^^ and to update the output with when their absolute difference 

is less than the threshold T. 

5 NUMERICAL EXPERIMENTS 

The standard 8 bit, gray-scale "Lena" image (size 512x512) is used as an 
example to test the usefulness of our filtering algorithm. We degraded it with various 
percentages of fixed-value (0 or 25 5) impulse noise. The proposed algorithm is compared 
with the median filtering and a-TMF algorithms, and their peak signal-to-noise ratio 
1 0 (PSNR)performances are listed in TABLE I. 



TABLE I 

Comparative Filtering Results in PSNR for Lena Image 
Corrupted with Different Amount of Fixed Impulse Noise 



15 



Algorithm" 


15% 


20% 


25% 


30% 


35% 


40% 


Median 


32.19 dB 


31.48 dB 


30.69 dB 


29.91 dB 


29.37 dB 


28.75 dB 


a-TMF 


32.09 dB 


31.30 dB 


30.39 dB 


29.28 dB 


28.38 dB 


27.49 dB 


VWTMF 


32.31 dB 


31.67 dB 


30.89 dB 


30.12 dB 


29.63 dB 


29.06 dB 


Median Switch 


35.20 dB 


33.87 dB 


32.91 dB 


31.84 dB 


30.99 dB 


30.29 dB 


a-TMF Switch 


36.04 dB 


34.93 dB 


33.97 dB 


32.59 dB 


31.67 dB 


30.72 dB 


VWTMF Switch 


36.34 dB 


35.13 dB 


34.29 dB 


33.15 dB 


32.24 dB 


31.43 dB 



* All algorithms are implemented recursively for optimal performance. 



25 Both direct and switch-based application of the filters are presented in TABLE I for 

comparison, and all algorithms are implemented reciu^i vely in a 3 x 3 window for optimal 
performance (The same iterative procedure is used for all the switching scheme based 
algorithms.) A 3-entry median basket (1=1) is used for both a-TMF and VWTMF 
algorithms. The A used in the VWTMF weight fiinction is 2 for images with 15%, 20%, 

30 25%, and 30% impulse noise, and is 3 with 35% impulse noise, and is 4.5 with 40% 

impulse noise. The threshold used for the switching schemes is 28. 

From TABLE I, it is easy to see that without the switching scheme, the a-TMF 
performs even worse than the median filter. However, it performs better than the median 
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filter when the switching scheme is used. This reflects the fact that although the a-TMF 
may not perform well in impulse noise removal, it is a good impulse detector. The 
VWTMF performs better than either the median filter or the a-TMF, whether the 
switching scheme is used or not. It is simple, robust and efficient. The VWTMF performs 
well in removing impulse noise, and is simultaneously a good impulse detector It is 
especially efficient for filtering highly impulse noise corrupted images. Figure 2 shows 
the original noise-firee image, the impulse noise corrupted image (40% impulse noise), 
and the filtered results for several algorithms. One can observe firom Figure 2 that even 
if there is no switching scheme employed, the VWTMF performs better than either the 
median filter or the a-TMF in terms of suppressing noise and preserving edges. The 
a-TMF performs even worse than the median filter when no switching scheme is 
employed. Many speckles still remain in the a-TMF filtered image. However, the 
VWTMF-based switching scheme provides a result that is almost the same as the original 
noise-firee image. 

CONCLUSIONS 

This paper presents a new fihering algorithm for removing impulse noise fi-om 
corrupted images. It is based on varying the weights of the generalized trimmed mean 
fiher (GTMF)continuously according to the absolute difference between the luminance 
values of selected pixels in the median basket and the median value. Numerical results 
show that the VWTMF is robust and efficient for noise removal and as_ an impulse 
detector. Although the VWTMF is only used for removing impulse noise in this paper, 
we expect that it also will be useful for removing additive noise. 
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A NEW NONLINEAR IMAGE FILTERING TECHNIQUE 
INTRODUCTION 

Images are often corrupted by noise that seriously affects the performance of 
various signal processing techniques, data compression, and pattern recognition. The goal 
of noise filtering is to suppress the noise while preserving the integrity of significant 
visual information such as textures, edges and details. Linear local averaging filters are 
essentially low pass filters. Because the impulse responses of the low pass filters are 
spatially invariant, rapidly changing signals such as image edges and details, cannot be 
well preserved. Consequently, impulse noise caimot be effectively removed by linear 
methods; nonlinear techniques have been found to provide more satisfactory results. 
Some of the most popular nonlinear filters are the median filter [1], and its various 
generalizations [2-8], which are well knovra to have the required properties for edge 
preservation and impulse noise removal. However, the median filter is not optimal since 
it is typically implemented uniformly across the image. It suppresses the true signal as 
well as noise in many applications. In the presence of impulse noise, the median filter 
tends to modify pixels that are not degraded. Furthermore, it is prone to produce edge 
jitter when the percentage of impulse noise is large. In order to improve the performance 
of the median filter, two median based filters, namely the a-trimmed mean (a-TM) [5-6] 
filter and the modified trimmed mean (MTM) [6] filter, which select firom the window 
only the luminance values close to the median value, have been proposed. The selected 
pixels are then averaged to provide the filtering output. Although superior to the median 
filter in some appUcations, these algorithms still have problems. In general, the MTM 
outperforms the median filter in removing additive noise at the cost of increased 
computational complexity, but not as well as the median filter in removing impulse noise. 
The a-TM filter is in general superior to the median filter as an impulse detector, but its 
performance in removing impulse noise is not so well. As is shown in the test example 
of this paper, the a-TM filter performs even worse than the median filter when no 
impulse detection techniques are employed. Nevertheless, when the level of impulse 
noise is high, the a-TM filter is not optimal for detection, since the selected pixels may 
have a large probability of being corrupted by impulse noise. Removal of additive noise 
is a problem for the a-TM and MTM filters because of not taking account of the central 
pixel in the window. 
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A new filtering technique is disclosed, using what we call a generalized trimmed 
mean (GTM) filter, which in general outperforms a-TM and MTM filters for images that 
are highly corrupted by impulse or additive noise. The GTM filter- is based on a 
generalization of the a-TM filter. In addition to the commonly used moving spatial 
5 window a symmetric median basket is employed to collect a predetermined 

number of pixels on both sides of the median value of the sorted pixels in the window. 
The luminance values of the collected pixels and the central pixel in the window are 
weighted and averaged to obtain an adjusted value G{m,n) for the central pixel (m,n). For 
the removal of additive noise, it is very usefiil to have the central pixel participate in the 

1 0 averaging operation because the probability of the luminance value of the central pixel 

to be the closest to the noise-fi-ee value is larger than for the other pixels. However, when 
impulse noise is presence, we usually set the weight of the central pixel to zero because 
the impulse-noise-conrupted pixels are independent to the noise-firee pixels. For images 
that are highly corrupted with impulse noise, it is in general effective to require the 

1 5 selected values in the median basket to have different weights in order that the impulses 

selected for the basket not affect the fihering output very much. Furthermore, in the 
procedure of impulse noise removal, the proposed filter is combined with a switching 
scheme [9] to produce an impulse detector to preserve the noise-firee pixels exactly, 
while providing an optimal approximation for the noise-corrupted pixels. Many impulse 

20 detection algorithms have been proposed [9- 1 2] . In [9], a median filter-based switching 

impulse detector is employed. The basic idea is to calculate the absolute difference 
between the median filtered value and the input value for each pixel. If the -difference is 
larger than a given threshold, the output is the median filtered value; otherwise, the 
output equals the input value. We do the same except that we replace the median filter 

25 with the GTM filter. In order to remove impulse noise effectively, we here propose a new 

iteration scheme which generally improves the performance of the filter We use both the 
output of the last iteration and the initial input as the input in the current iteration 
calculation. The GTM filter is applied to the output of last iteration to give an 
intermediate output. If the absolute difference between the initial input and the 

30 intermediate output is larger than a predetermined threshold 7, the current output is the 

intermediate output, otherwise, the current output is the initial input. In contrast to the 
traditional iteration technique, our iteration scheme does not use the output of the last 
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iteration to do the switching operation (the second step). 

GENERALIZED TRIMMED MEAN FILTER 

In the commonly-used median filtering procedure, the luminance value of a pixel 
is replaced by the median value in a neighboring spatial window 

<S>im,n)=([m-W, m+W]x[n-W, n-^W]) (1) 
The size of this moving square window is thus N= (2Pr+l)x(2FF+ 1). Let the median 
luminance value in this spatial window be denoted as 

M(m,n) = Median ({/( v")} e (D(/n,«) (2) 
where is the luminance value at pixel (ij). We reorganize the pixels in the window 
as a new list according to the ascending order of their luminance values- 
/ (-L(A^-l)/2j)<...</' {A)<r (0)</(i; <,.. 

<r (L(iv+i)/2j) (3) 

where /(O) is the median value in the neighborhood of pixel (m,n), i.e., / (0)=Af(m,n). 

The key generalization to median filtering introduced in the alpha-trinuned mean 
(a-TM) filter [5-6] is to design a symmetric median basket according to luminance value 
in order to combine a given number of pixels on both sides of the median value of the 
sorted pixels in the window. The collected pixels are then averaged to give the filtering 
output, as follows 

'""■•"'■2rj>,|''«- (4) 

where i=|_^J with 0^ ^ir^ 0.5. When ^0, it becomes the median filter, and when ^0.5, 
it becomes the simple moving average. In general, the ^2r-TM filter outperforms the 
median filter in detecting the impulse. However, its capability of removing the impulse 
noise is even worse than for the median fiUer when no additional procedure is employed 
(such as the switching scheme). When an image is corrupted by high levels of impulse 
noise, the a-TM filter does not perform well, since the pixels being selected for the 
median basket now have a large probability of being impulse corrupted, it is therefore 
unreasonable for the a-TM filter to have the pixels in the basket equally weighted. 
Another generalization to the median filtering is the so called modified trinuned mean 
(MTM) fiher [6]. In the MTM filter, a container C{m,n) is employed to select the pixels 
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whose luminance values are in the range of [M{m,n)'q, M(m,n)+q], with q being a 
predetermined threshold. The mean value of the selected values in the container is taken 
as the filtering output. - 

T(m,n) = Memmij)}\I(ij^ (5) 
5 Like the a-TM filter, the MTM fiher can also be reduced to the median filter (at q=0 ) 

or the simple moving average (at q=255 for 8bpp gray scale image). The MTM filter is 
useful for removing additive noise but does not perform as well as the median filter when 
impulse noise removal is required, since impulse noise corrupted pixels are independent 
of the noise-fi-ee pixels. 

1 0 There is still one problem left for the a-TM and MTM filters where additive noise 

removal is concerned; they do not take special account of the luminance value of the 
central pixel in the window. As is well known, the value of the central pixel, in general, 
has a larger probability of being the closest to its true value than those of all other pixels 
in the window. We therefore design the generalized trimmed mean (GTM) filter which 

1 5 improves the performance of both the a-TM and MTM filters. In the first step of the new 

filter in process, we employ the median basket to collect those pixels whose luminance 
values are close to the median value, in the same way as is done in the a-TM filter. The 
difference between our algorithm and the a-TM algorithm is that a weighted averaging 
is performed using the values selected for the median basket, as well as the value of the 

20 central pixel (In general, the central pixel is used in removing the additive noise.). Thus. 

L 

w,/(m,/i)+2^w,;'(0 
G(m.n)- 'f^ ' (g) 

where and iv/s are the weights for the central pixel and the pixels in the median basket 
25 respectively. When w=0 and all w/s are the same, it becomes the a-TM filter. Where 

filtering impulse noise corrupted images is concerned, we usually set ^^=0 because the 
amplitude and the position of the impulse is independent of the original true signal. 
Compared with the a-TM filter, the GTM filter is usually more efficient when highly 
impulse noise corrupted images are to be filtered. For such an image, the weight Wq for 
30 the median value is usually chosen to be higher than those of other pixels in the median 

basket because the median pixel usually has the least probability to be corrupted by 
impulse noise. Smaller weights for the pixels other than the median pixel can serve as an 
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adjustment for the filtering output, which is important in the iteration calculation. The 
problems of applying the a-TM filter to highly impulse noise corrupted images is that 
the weights for the pixels other than the median pixel are too large, which may give 
filtering results that are still highly corrupted. As explained above, because the GTM 
5 fiher includes the central pixel in the averaging operation, it is reasonable to expect that 

its performance in removing additive noise will be improved compared to the a-TM and 
MTM filters. 

For the removal of impulse noise, a switching scheme [9] based on our GTM 
filter is employed to detect the impulse noise corrupted pixels. The filtering output I(m,n) 
10 for a pixel (m,n) is generated by the following algorithm: 



UX^.nl |/,(/w,/i)-G(m.«)l<7' ^ 
^^'"'"^"\G(m,/i). |/,(/n./j)-G(m,/j)l>r 

(7) 

15 

where is the original input pixel value and 7 is the switching threshold used to test the 
difference between the original input and output of the GTM filter. 

a) If the difference is larger than T, it implies that the pixel differs significantly 
from its neighbors. It is therefore identified as a noise corrupted pixel, and is replaced by 

20 G(w,/i). 

b) If the difference is smaller than T, it implies the original pixel is ^iniirar to its 
statistical neighbors, and we identify it as noise firee, therefore retaining its original input 
value. 

For more seriously impulse-noise-comipted images, an iterative application of 
25 the above procedure is used to improve the filtering performance. The iteration 

processing can be summarized as 



30 



where /(m,n|r) is the system output at time /, G(m,n\t) is the intermediate output obtained 
by applying [6] to /0V|/-1) with {ij)eO{m,n). To initialize the algorithm, we set 
I{ij\0)-Ii{ij). The above procedure can be expressed compactly as 
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I(rn,n)\t) = I,(m,n)S(m,n)\t) + G(m,n)\t)[l - S(m,n)\t)] (9) 
where the step function S(m,n\t) is defined by 

SMoJ^ IAK-)-G(m,n|r)|<7 

5 

It is important for the iteration procedure to always compare G(m,n\t) with the initial 
-input /Xm,n) and update the filtering output with when their difference is less than 
the threshold T. It is not good to use |M) instead of /^m,/!) to either do the 
comparison or updating operation. 

10 

EXAMPLE APPLICATIONS 

To test the proposed filtering technique, the benchmark 8bpp gray-scale image, 
"Lena", size 512x512, is corrupted with different percentages of fixed value impulse 
noise (40% and 60%), and the Gaussian noise with peak signal-to-noise ratio (PSNR) 
15 1 8.82 dB respectively. ThesymmetricmovingwindowsizeisSxS, withaS-entrymedian 

basket used for both cases. The PSNR, mean square error (MSB) and mean absolute error 
(MAE) comparisons of several different filtering algorithms are shown in TABLE I for 
impulse noise and TABLE 11 for Gaussian noise. 



20 



25 



30 



35 



TABLE I. 

Comparative Filtering Results for Lena Image with Fixed Impulse Noise 



Algorithm* 


Impulse (40%) 


Impulse (60%) 




PSNR 
(dB) 


MSB 


MAE 


PSNR 
(dB) 


MSE 


MAE 


No Denoising 


9.29 


7659.43 


51.03 


7.54 


11467.21 


76.45 


Median (3x3) 


28.75 


86.79 


4.72 


23.38 


298.71 


7.25 


Median (5x5) 


28.15 


99.65 


5.21 


26.15 


157.65 


6.54 


a-TM (3 x3) 


27.49 


115.86 


6.05 


23.44 


294.41 


10.70 


Median Switch 
(3x3) [51 


30.29 


60.77 


2.54 


23.73 


275.26 


5.50 


a-TM Switch 
(3x3) 


30.72 


55.15 


2.33 


26.66 


140.31 


4.52 


GTM Switch (3x3) 


31.18 


49.55 


2.22 


27.57 


113.84 


3.98 



algorithms are iterated in the same way and the switching threshold T is 28. Three-entiy median basket 
are used in the a-TM and the new algorithms. The weights for our algorithm are w^^iyvQ-.w^iw^-l: 14: 1 : 0. 
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TABLE II. 

Comparative Filtering Results for Lena Image with Gaussian Noise 



Algorithm 


PSNR 
(dB) 


MSE 


MAE 


No Denoising 


18.82 


853.07 


23.71 


Median (3x3) 


27.60 


112.97 


7.72 


Median (5x5) 


27.39 


118.41 


7.53 


MTM (3x3) 


28.23 


97.74 


7.39 


a-TM (3x3) 


28.13 


100.10 


7.33 


GTM (3x3) 


28.38 


94.48 


7.04 



♦All algorithms are implemented recursively for optimal 
PSNR and MSE performance. Three-entry median basket 
are used in the last two algorithms. The weights for our 
algorithm are w w,: ^^=1:1:1:10. The threshold q for 
the MTM algorithm is optimized to 100. 



The fihering parameters are also shown in the TABLES. All algorithms are implemented 
recursively for optimal PSNR and MSE performance. The new iteration scheme proposed 
in this paper also has been implemented with a-TM filter for filtering of impulse noise. 
It is evident firom TABLE I that our GTM filter-based switching scheme yields improved 
results compared to both the median and a-TM filter-based switching schemes. In 
general, the a-TM filter-based switching scheme performs better than the median filter- 
based switching scheme. However, without the switching scheme, the a-TM filter 
performs even worse than the median filter for such highly impulse-noise-cormpted 
images, showing that although the a-TM filter is a good impulse detector, it is not good 
at removing impulse noise by itself. As shown in Figure 1(a), the original Lena image is 
degraded by adding 60% impulse noise. Figure 1(b) is the filtering result using the 
method of Sun and Neuvo [9] with a 3x3 window and switching threshold of r=28. One 
can observe both blur and speckles on the image; the 3x3 window is not suitable for 
applying the median filtering algorithms for such a high-noise image. These speckles can 
be removed by increasing the window size firom 3x3 to 5x5 but at the expense of even 
more blurring after many iterations. Figure 1(c) shows our tiltering result with the same 
switching threshold. It is seen that our algorithm yields improved results. Because our 
algorithm is implemented in a 3x3 window, it is possible to preserve more detail of the 
image than the algorithm of Sun and Neuvo [9] implemented in a 5x5 window with many 
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iterations. The remaining small number of speckles can be removed by filtering using a 
5x5 window once (with the same basket-weight vector) and then continuing to iterate 
using the3x3 window; see Figure 1(d). For the 40% impulse noise corrupted Lena image 
in Figure 2(a), the filtering performance is compared with commonly used median filters 
5 having different window size. In order to remove the speckles fi-om the image, the 

median filters must introduce some blurring and aliasing to the image (Figure 2(b) and 
2(c)), while our filtering result is significantly improved (Figure 2(d)) in comparison. For 
the filtering of highly impulse- noise-corrupted images, the perfomiance is usually 
improved when the median value is given a larger weight than other pixels in the median 
10 basket. TABLE II shows that it is reasonable to include the luminance value of the 

central pixel in the averaging operation for the removing of Gaussian noise. With a 
weight of the central pixel larger than those of other pixels in the median basket, 
improving iteration performance can be obtained in removing additive noise. 



15 CONCLUSIONS 

In this paper, we present a new algorithm for image- noise removal. A given 
number of significant luminance values on both sides of the median value of the sorted 
pixels in the neighboring window are bundled with the central pixel and weighted to 
obtain a modified luminance estimate (MLE) for the central pixel. For the removal of 

20 impulse noise, a threshold selective-pass technique is employed to determine whether the 

central pixel should be replaced by its MLE. A new iterative processing is designed to 
improve the performance of our algorithm for highly impulse noise corrupted images. 
For effectively removing additive noise, it is usefiil for the filtering technique to include 
the value of the central pixel as one of the values being averaged. Numerical experiments 

25 show that our technique is simple, robust and efficient, and leads to significant 

improvement over other well-known methods. 
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BIOMEDICAL SIGNAL PROCESSING USING A NEW CLASS OF WAVELETS 
INTRODUCTION 

In general, signal filtering and processing may be regarded as a kind of 
approximation problem with noise suppression. According to DAF theory [11, a signal 
approximation in DAF space can be expressed as 



where the SJx) is a generalized symmetric Delta functional sequence. We choose it as 
a Gauss modulated interpolating shell, or the so-called distributed approximating 
functional (DAF) wavelet. The Hermite-type DAF wavelet is given in the -following 
equation [2]. 




The function //^is the Hermite polynomial of even order, 2n. The qualitative behavior 
of one particular Hermite DAF is shown in Figurel. 

Different selections of interpolating shells result in different DAFs. Theoretically, 
this kind of functional can be regarded as the smoothing operator or the scaling function 
in wavelet theory. It can be used to generate the corresponding wavelets (differential 
functional) for signal analysis. The discrete wavelet transform is implemented using 
filterbanks. 

Additionally, the continuous type of DAF wavelet is used to construct DAF 
wavelet neural nets. The DAF wavelet neural nets possess a modified form the 
conraionly used DAF approximation, given as 



i(x) = ZKO<ya(Jc-jf/) (3) 

The weights w{i) of the nets determine the superposition approximation ^jc) to the 
original signal g{x)eL\R). It is easy to show that the weights of the approximation nets, 
w(i), are closely related to the DAF sampling coefficients g(x^). If the observed signal is 
limited to an interval / containing a total of //discrete samples, /={0, 1, 7V-1}, the 
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square error of the signal is digitized according to 

£A^'i\sin)'gin)f (4) 

5 This cost function is commonly used for neural network training in a noise-jfree 

background and is referred to as the minimum mean square error (MMSE) rule. 
However, if the observed signal is corrupted by noise, the network produced by MMSE 
training causes an unstable reconstruction, because the MMSE recovers the noise 
components as well as the signal. MMSE may lead to Gibbs-like imdulations in the 

1 0 signal, which is especially harmful for calculating accurate derivatives. Thus, we present 

a novel regularization design of the cost function for network training. It generates edge- 
preserved filters and reduces distortion. To achieve this, an additional smooth derivative 
term, is introduced to modify the original cost function. The new cost function is 

15 =ZUW-iWl'-^^J«[^l'^ (5) 

k 

The factor Jl introduces a compromise between the orders of approximation and 
smoothness. To increase the stability of the approximation system further, an additional 
constraint in state space is taken to be 



20 



^"^^li;^ (6) 
Thus the complete cost function utilized forDAF wavelet neural net training is given by 



25 



(7) 



MAMMOGRAM ENHANCEMENT 

Medical imaging, including computed tomography (CT), magnetic resonance 
30 imaging (MRI), X-ray mammography, ultrasound and angiography, includes some of the 

most useful methods for diagnosis. In particular. X-ray mammograms are widely 
recognized as being the most effective method for the early detection of breast cancer.- 
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Major advances in mammograms have been made over the past decade, which have 
resulted in significant improvements in image resolution and film contrast without much 
increase in X-ray dosage. In fact, mammogram films have the highest resolution in 
comparison of various other screen/film techniques. However, many breast cancers 

5 cannot be detected based on mammogram images because of the poor visualization 

quality of the image. There are also many false positives that result in substantial stress 
to patients and their families. Both types of difficulties are substantially due to the minor 
differences in X-ray attenuation between normal glandular tissues, benign formations and 
malignant disease, which leads to the low-contrast feature of the image. As a result, early 

1 0 detection of small tumors, particularly for younger women who have denser breast tissue, 

is still extremely difficult. 

Mammogram image processing has recently drawn a great deal of attention [3-8]. 
Most work focuses on noise reduction and feature enhancement. The statistical noise 
level is relatively low for images obtained by modem data acquisition techniques. 

1 5 Therefore feature enhancement is more essential for mammogram quality improvement. 

Since an image noise reduction algorithm was reported earlier [9, 10], we shall focus on 
image enhancement in this work. In our opinion, there are two basic approaches for 
mammogram feature improvement. One type of method changes the image spectrum 
distribution so as to increase image edge densities. As a result, the image appears sharper. 

20 The other approach is the so-called image windowing, in which linear or nonlinear 

grayscale mappings are defined so as to increase/compress the image gradientdensity. 
In an earlier work, one of present authors realized that the fi-equency responses of wavelet 
transform subband filters are not uniform. A magnitude normalization algorithm was 
proposed to account for this non-unifonn feature. This idea was later extended to group 

25 normalization for the wavelet packet transform [11, 12] and to visual group 

normalization for still-image noise reduction [ 1 3] by the present authors. In this work we 
fiirther extend this idea to achieve image enhancement. 

Our first approach is based on a rearrangement of the image spectrum distribution 
by an edge enhancement normalization of the wavelet coefficients. Our second technique 

30 utilizes a multiscale fimctional to obtain a device-adapted visual group normalization of 

the wavelet coefficients. Numerical experiments indicate that our normalization 
approach, in combination with the use of the DAF wavelets [13], provides excellent 
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enhancement for low quality mammogram images. 

To test our new approaches, low-contrast and low quality breast mammogram 
images are employed. Mammograms are complex in appearance and signs of early 
disease are often small or subtle. Digital mammogram image enhancement is particularly 
5 important for aiding radiologists in long distance consultation, image storage and 

retrieval, and for the possible development of automatic detecting expert systems. Figure 
2(a) is an original 1024x1024 side-view breast image which was obtained from the 
Mammographic Image Analysis Society (MIAS) and has been digitized to 8 bits of gray 
scale. The enhanced image result is depicted in Figure 2(b), and shows significant 
1 0 improvement in image quality. 

ECG FILTERING 

Automatic diagnosis of electrocardiogram (ECG or EKG) signals is an important 
biomedical analysis tool. The diagnosis is based on the detection of abnormalities in an 

1 5 ECG signal. ECG signal processing is a crucial step for obtaining a noise- free signal and 

for improving diagnostic accuracy. A typical raw ECG signal is given in Figure3. The 
letters P, Q, R, S, T and U label the medically interesting features. For example, in the 
normal sinus rhythm of a 12-lead ECG, a QRS peak follows each P wave. Normal P 
waves show 60-100 bpm with <10% variations. Their heights are <2.5nmi and widths 

20 <0. 1 1 s in lead II. A normal PR interval ranges from 0. 1 2 to 0.20s (3-5 small squares), A 

normal QRS complex has a duration of <0. 1 2s (3 small squares). A corrected QT interval 
(QTc) is obtained by dividing the QT interval with the square root of the preceding R-R' 
interval (normally QTc=0.42s). A normal ST segment indicates no elevation or 
depression. Hyperkalaemia, hyperacute myocardial infarction and left bundle can cause 

25 an extra tall T wave. Small, flattened or inverted T waves are usually caused by 

ischaemia, age, race, hyperventilation, anxiety, drinking iced water, LVH, drugs (e.g. 
digoxin), pericarditis, PE, intraventricular conduction delay (e.g. RBBB) and electrolyte 
disturbance [20]. 

An important task of ECG signal filtering is to preserve the tme magnitudes of 
30 the P, Q, R, S, T, and U waves, protect the true intervals (starting and ending points) and 

segments, and suppress distortions induced by noise. The most common noise in an ECG 
signal is AC interference (about 5OH2-6OH2 in the frequency regime). Traditional- 
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filtering methods (low-pass, and band-elimination filters, etc.) encounter difficulties in 
dealing with the AC noise because the signal and the noise overly the same band. As a 
consequence, experienced doctors are required to carry out time-consuming manual 
diagnoses. 

Based on a time varying processing principle, a non-linear filter [ 1 4] was recently 
adopted for ECG signal de-noising. Similar to the selective averaging schemes used in 
image processing, the ECG is divided into different time segments. A sample point 
classified as "signal" is smoothed by using short window averaging, while a **noise" 
point is treated by using long window averaging. Window width is chosen according to 
the statistical mean and variance of each segment. However, this c^culation is 
complicated and it is not easy to select windows with appropriate, lengths. The 
regularized spline network and wavelet packets were later used for adaptive ECG 
filtering [12, 15], which is not yet efficient and robust for signal processing. In our 
present treatment, regularized D AF neural networks are used to handle a real-valued ECG 
signal. We utilize our combined group-constraint technique to enhance signal 
components and suppress noise in successive time-varying tilings. 

The raw ECG of a patient is given in Figure 4(a). Note that it has typical thorn- 
like electromagnetic interference. Figure 4(b) is the result of a low-pass filter smoothing. 
The magnitudes of the P and R waves are significantly reduced and the Q and S waves 
ahnost disappear completely. The T wave is enlarged, which leads to an increase in the 
QT interval. Notably, the ST segment is depressed. Such a low-pass filtering result can 
cause significant diagnostic errors. Figure 4(c) is obtained by using our DAF wavelet 
neural nets. Obviously, our method provides better feature-preserving filtering for ECG 
signal processing. 
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VISUAL GROUP NORMALIZATION USING G AUSSI AN-LAGRANGE D AFWs 
INTRODUCTION 

Distributed approximating fiinctionals (DAFs) were introduced as a powerfiii grid 
method for quantum dynamical propagations [1]. DAFs can be regarded as scaling 
functions and associated DAF-wavelets are generated in a number of ways [2]. DAF- 
wavelets are smooth and decaying in both time and frequency domains and have been 
used for numerically solving linear and nonlinear partial differential equations with 
extremely high accuracy and computational efficiency. Typical examples include 
simulations of 3D reactive quantum scattering and 2D Navier-Stokes equation with non- 
periodic boundary conditions. The present work extends the DAF approach to image 
processing by constructing interpolating DAF-wavelets[3]. An earlier Group 
normalization (GN) technique [4] and human vision response [5] are utilized to normalize 
the equivalent decomposition filters (EDFs) and perceptual luminance sensitivity. The 
combined DAF Visual Group Normalization (VGN) approaches achieve robust image 
restoration result. 

INTERPOLATING DAF WAVELETS 

Interpolating wavelets are particularly efficient for calculation since their 
multiresolution spaces are identical to the discrete sampling spaces. Adaptive boundary 
treatments and irregular samplings can be easily implemented using symmetric 
interpolating solutions. We design the interpolating scaling function as an interpolating 
Gaussian-Lagrange DAF (GLDAF) [6] 



^ (1) 



where W^x) is selected as a Gaussian window since it satisfies the minimum-frame- 
bound condition in quantum physics. Quantity a is width parameter. Pj^x) is the 
Lagrange interpolation kernel. DAF dual scaling and wavelet functions are constmcted 
for perfect reconstruction [3]. The Gaussian window efficiently smoothes out the 
oscillations, which plague many wavelet bases. As shown in Figure 1, both DAF 
wavelets and their dual partners display excellent smoothness and rapid decay comparing 
with the popular B97 wavelets. 
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VISUAL GROUP NORMALIZATION 

Wavelet transform is implemented by a tree-structure filtering iteration. The 
coefficients can be regarded as the output of a single equivalent decomposition filters 
(EDF). As shown in Figure 2(a) and Figure 2(b), EDF responses without normalization 
are much different. Clearly, the decomposition coefficients cannot exactly represent the 
actual signal strength. As an adjusted solution, wavelet coefficients, C„(k), in block m 
should multiply with a magnitude scaling factor, yi„. We define this factor as the 
reciprocal of the maximum magnitude of the EDF fi^equency response [4] (where LC„is 
m-th EDF response). 

''- = sup{|4..(-)l} " = (2) 



The normalized EDF responses of DAF and B97 wavelets are shown in Figure 
2(c) and Figure 2(d), respectively. It is easy to find out that DAF 's possess smaller 

15 sidelobes, which induce less fi-equency leakage distortion. 

An image is human-vision-dependent source. Using a just-noticeable distortion 
profile, we can efficiently remove the visual redundancy as well as normalize the 
coefficients in respect to perception importance. A mathematical model for "perception 
lossless" matrix Y„ has been presented in [5] and is used as perceptual normalization 

20 combined with the EDF magnitude normalization. (Noted here we use A„ for magnitude 

normalization not the wavelet "basis fimction amplitude" in [5], because the digital image 
decomposition is completely operated using filter banks) We denote the combination of 
the two normalized operations as Visual Group Normalization (VGN). 

25 EXPERIMENTAL RESULTS 

To test our approaches, two benchmark 512x512 Y-component images are 
employed (Barbara with much texture and high fi-equency edges, while Lena with large 
area of flat region). The PSNR comparison results are shown in Table I, while the 
perceptual quality comparison of Lena is shown in Figure4. It is evident that our 

30 technique yields better performance. 
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NONLINEAR QUINCUNX FILTERS 
INTRODUCTION 

The possible image noises include photoelectric exchangej photo spots, the 
error of image communication, etc. The noise causes the speckles, blips, ripples, 
bumps, ringings and aliasing that not only affects the visual quality, but also 
degrades the efficiency of data compression and coding. Developing de-noising 
and smoothing technique is important. 

The popularly used noise model is Gaussian, since it is easy for linear 
(stationary) analysis. In practice, the further physical environments are more 
accurately modeled as impulsive, which characterized by heavy^tailed non- 
Gaussian distributions. Such noises include atmospheric noise (due to a lightening 
spike and spurious radio emission in radio communication), ice cracking and 
aquatic animal activity in sonar and marine communication, and relay switching 
noise in telephone channels. Moreover, a great variety of synthetic noise sources 
(such as automatic ignitions, neon lights, and other electronic devices) are also 
impulsive. Impulsive interference may be broadly defined as signal corruption that 
is random, sparse, and of high or low amplitude relative to nearby uncorrupted 
signal values. The performance of filter systems developed under the assumption 
of Gaussian condition are severely degraded by the non-Gaussian noise due to 
large deviations from normality in the tails. Independent on the physical 
environment, relatively infrequently occurring and non-stationaiy are the 
important features of impulse noise that we can not obtain an accurate statistical 
description. Therefore, it is impossible to design an optimal linear filtering systems 
based on the generalized likelihood ratio (GLR) principles. Consequently, by first 
suppressing the impulsive component and then operating on the modified signal 
with traditional linear system, near-optimal performance is obtained. The non- 
Gaussian nature of the impulse noise dictates that the suppression filter be 
nonlinear and robust due to the presence of impulses . Traditional image processing 
is always defined on the whole space (time) region, which does not localize the 
space (time)-frequency details of the signal. The mean error may be improved, but 
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the averaging process will blur the silhouette and finer details. 

New research shows that non-Gaussian and non-stationary characterize the 
human- visual-response (HVS) based image processing. Human visual perception 
is more sensitive to image edges which consist of sharp-changes of the 
neighboring gray scale because it is essentially adaptive and has variable lenses 
and focuses for different visual environments. To protect edge information as well 
as remove noise, modem image processing techniques are predominantly based 
on non-linear methods. Before the smoothing process, the image edges, as well as 
perceptually sensitive texture must be detected. Some of the most popular 
nonlinear solutions are the median filter [1] and its generalizations [2, 3], that 
possess the required properties for impulse noise removal. However, the median 
filter is not optimal since it is typically implemented uniformly across the image. 
It suppresses the true signal as well as the noise in many applications. In the 
presence of impulse noise, the median filter tends to modify pixels that are not 
degraded by noise. Furthermore, it is prone to produce edge jitter when the 
percentage of impulse noise is large. 

A significant characteristic of the impulse noise is that only a portion of the 
pixels are corrupted and the rest are totally impulse-noise-fi:ee. The key point for 
filter design is the need to preserve the noise-free pixels exactly, while providing 
an optimal approximation for the noise-corrupted pixels. An impulse detector is 
required to determine the noise-comxpted pixels [4-8]. In [4], a median filter- 
based switching scheme is used to design the impulse detector. The basic idea is 
to calculate the absolute difference between the median filtered value and the 
original input value for each pixel. If the difference is larger than a given 
threshold, the output is the median filtered value; otherwise, the output is the 
original input value. However, using a single median filtered value can not 
increase the detection probability of impulse noise. In addition, median estimation 
is not optimal for high-percentage impulse-degraded image restoration.. In this 
paper, we propose a so-called "median radius" to design a collecting basket. The 
"basket members", whose luminance values are close enough to median value, are 
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weighted averaging to generate an adjustable impulse detector (optimal estimator). 
Noted here our technique is not the previously presented a-trim solutions [2]. For 
<2r-trim filtering, the median is calculated at first, then the pixels in a neighboring 
window are ordered according to the absolute difference from the median value 
(from minimum to maximum). The first three (or several) pixels (close to median) 
are weighted averaging to substitute the observed pixel luminance. In our method, 
we only collect the weighted pixels which are within the "median radius" (a-trim 
member may exceed this domain) for averaging. Additionally, a switching 
threshold is introduced to determine whether the observed pixel need filtering or 
not. Only the noise-corrupted one is changed. 

To improve the filtering performance, a "quincunx moving window" is 
presented to define the arbitrary shaped neighborhood (traditional window is the 
trivial square shape). The research in neurophysiology and psychophysical studies 
shows that the direction-selective cortex filtering mimics the human vision system 
(HVS). These studies have found a redial frequency selectivity that is essentially 
symmetric with band widths nearly constant at one octave [12]. This feature is 
much like the 2-D quincunx decomposition. Theoretical analysis and simulation 
show that our quincunx filtering technique is quite robust and efficient for 
different noise background. The paper is organized as follows: In Section II, the 
mathematical model of noise is described. In Section III, the imag« processing 
technique based on a quincunx filter is studied. Section IV concerns the arbitrary 
shape extension of the quincunx filter. A simulation is given in Section V. Section 
VI contains the conclusion. 

NOISE MODEL 

Additive Random Noise 

For a 2-D digital image, we assume the noise-free image value at pixel 
(m,n) is s{m,n), and I{m,n) is the observed image representation. If the observation 
model is a random noise cormpted case, the pixel value can be depicted as 
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I(m,nys(m,n)^w{m,n) ( 1 ) 

where w(m,n) is the noise components, and its magnitude is random. Normally we 
assume w(m,n) is a Gaussian process with zero mean and standard deviation <t^.. 
For a non-stationary process, cr^ is time-varying. The important characteristic of 
such noise is that all pixels may be degraded and the noise amplitude is 
statistically non-uniform. 

IMPULSIVE NOISE 

Normally, impulse noise is a result of a random process that the value of 
the cormpted pixel is either the minimum or a maximum value of ^ display 
instrument. The overall noise character could be positive (maximum), negative 
(minimum) or mixture (salt and pepper). The characteristic of such kind of noise 
is that the pixels are degraded with probability p. The noise model is expressed as 



v(m,/i) = 



e(/w,n), with p 
I{mji), with 1 - p 



where e(m,n) is a binary random number. For an 8-bit gray scale image 
representation (0=minima; 255=maxima), its value is 

[0, with pq 

^^'"•''^ = 1255. withl-po . (3) 

The probability of the black (minimum) value is and the probability of the white 
(maximum) value is l-p^. Normally for the positive impulse case, j7o=0; for the 
negative impulse case, p^^l; and for the salt and pepper, p^'^lll. Any alternation 
style can be obtained by different selection of the probability p^. 

More complex impulsive noise models are generated by a random 
magnitude degradation selection. The random value impulse noise array v{m,n) is 
represented as 

v(m,n)=z{m,n)r(m,n) (4) 
where the impulse generation r{m,n) is a random process representing an ever-present 
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impulse component with standard deviation a^, and z{mM) is a switching sequence of 
ones and zeros. Impulse noise seriously affects the performance of various signal- 
processing techniques, e.g. edge detection, data compression, and pattern recognition. A 
median filter is the commonly used nonlinear technique for impulse noise removal. 

Assume we have fixed a median window (possibly with any size) for median 
filtering. Let N^ ^ denote the number of impulse-corrupted pixels in median window 
0(/72,«) of node (m,«). If the window size is (conventionally tfie 3x3 or 5x5 square 
window), then it is known that the error measure is 



where median estimation probability ^A^^ed^s{m,n)]=Vx[N^ <{WA)ll] is given in [5]. 
It is easy to conclude that when the corruption probability p is much higher, most 

15 possibly that N^ >{W-\)I2. This imphes that the convincing probability (CP) of the 

median fihering (Pr[x„^=5(m,/2)]) is very small (e.g. with small probability, the output 
of a median filter is close to the original noise-fi-ee value). In other words, a traditional 
median filter is invalid for highly noise-corrupted image processing. For a uniformly 
noise degraded background, no matter what size the selective window is, statistically the 

20 performance of median filtering cannot be optimized (because the probability 

Pr[A''„,„<(Jr-l)/2] is the fixed statistically). Our objective is to improve the performance 
of nonlinear filtering and reduce the error measure above. 

NONLINEAR FILTERS 

25 In the median filtering procedure, the luminance value of a pixel is replaced by 

the median value in a neighboring spatial square window 

(l)(m,n)=([wi-/ni, /w+m2]x[n-/2„ n-^n^) (6) 
The size of this moving rectangular wmdow is A^=(/n,+m2+l)x(«i+n2+l). The 
conventional trivial windows are 3x3 or 5x5. Let the median luminance .Value in this 
30 spatial window be denoted as 

Ar(/72,/i)=Median{/(/w ;« ')\{m ;n OeO)} (7) 
where I{m ) is the luminance value at pixel (m We reorganize the pixels m the 
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window as a new list according to the order of their luminance value. 

/^(-L(Ar-l)/2j)<. . ,<I\0)<. . .<I\Ni(N-^iy2j) (8) 
where 7^(0) is exactly the median value in the neighborhood of pixel 

r(0)^M(m.n) (9) 
5 The key generalization to median filtering introduced in this paper is to design 

a "basket" according to luminance value in order to combine a group of pixels whose 
luminance levels are close to the median value of the window <J>(m,n), For each entry in 
this basket, a weighted average scheme is utilized to generate an adjusted median value 
as 

10 Z^im) 

H^i ' (10) 

where Q, is the set of pixels whose luminance values are close to median value in the 
window. The different design of our method from the a-trimmed filter is that we 

15 introduce an adjustable basket. The median distance (MD) A is defined as the absolute 

luminance difference between the observed value and the median filtered value M{m,n). 

A {m,n) = \I{m,nyM{m,n)\ (11) 
Only the pixels whose median distances are within a median radius (MR) y (e.g. {m,n)< 
y, Y>0), are selected as the "basket member" in Q for weighted averaging. Otherwise, 

20 their weights are set to zero. Normally, Wq is larger than or equal to the other, weights 

W^, />0. We call D{m,n) the modified luminance estimate (MLE) for pixel (m,n). Note 
that when the basket Q contains only one pixel f(0) (or the median radius 7=0), the filter 
is identical to the median filter, i.e., the median filter is a special case of our presented 
nonlinear filters. 

25 However, the modified luminance estimate (MLE) is not the ultimate filtered 

value to substitute the observed pixel. A switching scheme [2] based on MLE is 
employed to detect the impulse noise. The corresponding filtering output I{m,n) for a 
pixel (m,n) is generated by the foUov^dng algorithm: 

Ui{m,n), |/,(m,/i)-/)(m,n)l<r 
^ [Dim^), \Ii{m,n)-D{m,n)\>T ' (12) 
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where 1^ is the original input image. A threshold Tis used to test the difference between 
the original pixel value and the MLE value. The difference between the observed value 
I{m,n) and the adjusted median value D{m,n) from the basket can be regarded as the 
visible difference predictor (VDP). 

a) If the difference is larger than the threshold, it implies that the pixel differs 
significantly from its neighbors. It is therefore identified as a noise corrupted pixel, and 
is replaced by 

b) If the difference is smaller than the threshold, it implies the original pixel is 
similar to its statistical neighbors, and we identify it as noise free, therefore retaining its 
original value. For more seriously noise-corrupted images, an iterative application of the 
above procedure is required to obtain satisfactory performance. The iteration processing 
can be depicted as 



/(m./i|/) = 



A (w. nl I (ah. n) - Z)(m. /iU-I)|<r ^^3) 
D{m,rt|/-l)J/y(m.n)-D(m,f7U-l)|>r ' 



where I{m,n\i) is the system output at time /, D(m,n\t-\) is the MLE value of pixel 
at time t-\. To initialize the algorithm, we set /(m,n|0)=/X/w,/2). The above 
procedure can be simply expressed as 

/(w. « 1 0 = // (m, /))5(m. w I /) + D(m, « | / - l)[I - 5(m, /i | /)] , (14) 

where the step function S(m,n\t) is defined as 

c/ . X _ I^' I ^^^'"'"^ " U - 0 1< 

l/,-Kn)-D(m.n|/-l)t^r' ^^^> 

and it determines which value will be assigned to the pixel 

The proposed nonlinear filtering includes two steps; the first step is to obtain the 
adjusted reference value MLE of the observed pixel using median distance-based 
algorithm, and the second step is utilizing an active switching process to determine if the 
observed pixel is substituted by the MLE or not. (Noted for the traditional median 
filtering or a-trimmed solution, the pixel is only substituted by the reference value 
without any switching process.) A special nonlinear process is designed to switch 
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between the original pixel and MLE, depending upon the identification of the nature of 
the pixel. If the pixel is determined as noise-free, the original value should be preserved. 
Otherwise, it is replaced by the regressively weighted-median value. 

Our filtering is like Olympic scoring procedure, the lowest score and highest 
scores (outside the median radius) are removed and the remaining scores are then 
averaged to obtain the evaluation of the participant. In our approach, we remove several 
of the lowest luminance values and several of the highest luminance values from the 
pixels in the window. The remaining luminance values are then weight-averaged to give 
the filtering output for the pixel. We may consider the weighted average as assigning 
different referees different weights in scoring the participants. 

Theoretically, the MLE generation increases the probability 

PrK,,=5(m,/2)]=Pr[Ar„„<(PF-l)/2] (16) 
When the size of median basket is set to M, it equals to that the median window size W 
increases M times as MW, but with the same noise corrupted number „. In this case, 
the detection probability becomes 

PrK^=5(m,rt)]=Pr[A^„,„<(fWkf-l)/2] (17) 
This implies that the modified filtered output is more likely equal to the original noise- 
free value, 

QUINCUNX EXTENSION 

As shown in Figure 1, the traditional median or averaging filter considers the 
pixels in a square region. The most popular size is a 3x3 or 5x5 window, Avhich are used 
in the most popular papers and software packets (Photoshop and Lview, etc) on image 
processing. The filtering method or algorithm is applied in the square region (black 
pixels). One disadvantage of square window processing is that it can not adaptive to 
different noisy background and exist biases along the coordinate directions, or the spatial 
distance. This may cause visual distortion. 

Actually, besides the basic square window, the median basket (or the operation 
region) possesses numerous extended solutions. For exaniple, the 5x5 extension is shown 
in Figure 2. The traditional square shape approach gives preferential treatment to the 
coordinate axes and only allows for rectangular divisions of the frequency spectrum. The 
symmetry axes and certain nonrectangular divisions of the frequency spectrum 
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correspond better to the human vision system (HVS). These are typically concerned with 
two and three dimensions, as the algebraic conditions in higher dimensions become 
increasingly cumbersome. The predominant advantages of quincunx filtering is that there 
is very little biasing along coordinate directions, the sampling is nearly isotropic; i.e. 
5 there is no mixing of different scales; computationally excellent compression of 

discontinuous functions; and a simple representation of operators. In the 5x5 region, the 
basket size (black dot number) can be (1, 5, 9, 13, 17, 21, and 25). Thus we will have 
more alternation to offset the influence of over-smoothing caused by large window (5x5) 
processing. 

10 Moreover, some specially designed quincimx windows are conductive to remove 

the particular noise along different spatial direction, such as the quantization and 
thresholding noise of transform filtering and coding (discrete cosine transform (DCT) and 
wavelet transform (WT), etc.). Because the high-dimensional transforms (such as 2D 
DCT and wavelet transform) are usually the tensor products firom IDtransform (filtering) 

1 5 along different spatial directions, the coefficient error in the transform domain causes the 

distortion in the physical domain with the shape of the basis fimction (2D cosine and 
wavelet fimction). It usually looks like a small "cross" impulse. Even for the complicated 
non-separable wavelet transform case, the distortion is just a "rotated cross". Some 
quincunx windows will match the exact correlation characteristics of transform noise, as 

20 well as obtain the optimal performance. These filters can be utilized as a post-processor 

after the transform-based image processing and thresholding techniques. In the next 
chapter, we will report our quincunx post-processing results for a wavelet'thresholding 
technique, 

25 EXPERIMENTAL RESULTS 

We now present the results of computer simulations to demonstrate the 
effectiveness of our proposed techniques. Two objective criteria, the signal-to-noise ratio 
(SNR) and signal-to-impulse ratio (SIR) are used to evaluate and compare the 
performance. The signal-to-noise ratio, which is given by 

30 



SNR = lOlogiof ff ^i^^^^-^^l 



(18) 
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is used to evaluate the overall performance of the proposed preprocessor including 
prediction, nonlinearity and reconstruction capabilities. Another useful quantity, the 
signal-to-impulse ratio (SIR), is given by 

•5'^ = ioiog^^j (19) 

The SNR rule is derived from the traditional image quality criterion, which is 
characterized by a mean square error (MSE). It possesses the advantage of a simple 
mathematical structure. For a discrete signal {s(n)} and its approximation {^n)}, «=0, 
...,N, the MSE can be defmed to be 

MSE = l-'Z[sin)-s{n)f (20) 

However, the MSE based evaluation standard, such as peak signal-to-noise ratio (PSNR), 

PSNR =log[(255x255)/MS£] (2 1 ) 

can not exactly evaluate the image quality if one neglects the effect of human perception. 
The mixumnrnMSE rule will cause undulations of the image level and destroy the smooth 
transition informa.tion around the pixels. Therefore, we use both objective and subjective 
standards to evaluate our presented filtering results. 

The benchmark 8bpp gray-scale images are corrupted by additive impulse noise 
and Gaussian noise to test the proposed filtering technique. The practical symmetric 
quincunx windows are selected as shown in Figure 3. The peak signal-noise-ratio 
(PSNR), mean square error (MSE) and mean absolute error (MAE) comparison of 
different filtering algorithms for both images are shown in TABLE I (The amount of 
noise and the filtering parameters are also shown in TABLE I.). It is evident that our 
filtering-based switching scheme yields improved results compared to the median 
filtering-based switching scheme. In addition, the quincunx extension improves the 
filtering performance of the nonlinear processing. When noise probability is lower (20%), 
5-point diamond (or cross) quincunx filter possesses the better performance. When noise 
probability is higher (60%), 13-point diamond quincunx filter is testified better. The 9- 
point square window is suitable for 40% noise-degraded case. The perceptual quality of 
our algorithm is shown in Figure 4. The original Lena image is degraded by adding 40% 
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impulse noise in Figure 4(a). Figure 4(b) is the filtering result using our nonlinear filter. 

Wavelet noise is a special kind of additive noise generated by coefficient 
quantization error or thresholding error. Because the restored image is regarded as the 
linear or quincunx combination of different wavelet functions (scaling arid translation), 
5 the noise in the physical domain will be "random wavelets". For example, in a tensor- 

product wavelet transform case, the wavelet noise has a cross shape. Our quincunx filter 
can be utilized as a post-processor for a wavelet de-noising technique. As shown in 
Figure 5, Figure 5(a) is the Gaussian noise-degraded Lena. Figure 5(b) is the denoising 
result using DAF wavelet thresholding technique. Figure 6(c) is the one-time nonlinear 
10 quincunx restoration combined with wavelet thresholding. The result has- higher visual 

quality and 0.65dB PSNR improvement. 
CONCLUSIONS 

In this paper, we present a nonlinear quincunx filter for impulse or mixed noise 
removal. A specially designed "median basket" based on median distance is used to 

15 collect the "basket members" for calculating the modified median estimate (MLE). A 

switching scheme is used to detect the impulse noise and preserve the noise-firee pixels. 
Arbitrary shape quincunx windows are introduced to improve the visual filtering 
performance of our nonlinear filter. The quincunx version takes account of the different 
correlation structure of the image along different spatial directions, based on the human 

20 vision system (HVS). Iterative processing improves the performance of our algorithm for 

highly corrupted images. Numerical simulations show that the quincunx "filtering 
technique is extremely robust and efficient, and leads to significant improvement in 
different noise-degraded case. The method can be combined with any restoration 
technique to constitute a robust restoration method (for example as the post-processor for 

25 wavelet thresholding techniques). We compared the performance of these techniques, 

both subjectively and quantitatively, with the median filter and two of its recently 
proposed variants. Special attention was given to the ability of these methods to preserve 
the fine image details, such as edges and thin hues. In the experiments, our filtering gave 
the best results. 
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VISUAL MULTIRESOLUTION COLOR IMAGE RESTORATION 
INTRODUCTION 

Images are often contaminated by noise. Generally, the - possible noise 
sources of images are photoelectric exchange, photo spots, imperfection of 
communication channel transmission, etc. The noise causes speckles, blips, 
ripples, bumps, ringing and aliasing for visual perception. These distortions not 
only affect the visual quality of images, but also degrade the efficiency of data 
compression and coding. De-noising and restoration are extremely important for 
image processing. 

The traditional image processing techniques can be classified into two 
kinds: linear or non-linear. The principle methods of linear processing are local 
averaging, low-pass filtering, band-limit filtering or multi-frame averaging. Local 
averaging and low-pass filtering only preserve the low-band components of the 
image signal. The original voxel is substituted by an average of its neighboring 
voxels (within a square window). The mean error may be improved but the 
averaging process blurs the silhouette and details of the image. Band-limited filters 
can be utilized to remove the regularly appearing dot matrix, texture and skew 
lines. But they are useless for irregularly distributed noise. Multi-frame averaging 
requires the images be still, and the noise distribution be stationary, which is not 
the case for motion picture images or for a space (time)-varying.noisy background. 
Generally, traditional image processing is always defined on the whole space 
(time) region, which cannot localize the space (time)-frequency details of a signal. 
New research evidence shows that non-Guassian and non-stationary processes are 
important characteristics for the human visual response. 

Moreover, traditional image quality is evaluated by the mean square error 
(MSE), which possesses the advantage of a simple mathematical structure. For a 
discrete signal {s(n)} and its approximation {sfn)},h=Oy ...,A^-1, the.A/iSEcanbe 
defined as 

(1) 
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However, a MSE based measure, (such as P5A«=log[(255x255)/MS'£]), does not 
exactly evaluate image quality that consistent with human perception. The 
minimum MSE rule may cause serious oscillatory (no monotonic) behavior in the 
convergence process, which will destroy the original smooth transition-around the 
pixels. Commonly used regularization methods, such as regularized least squares, 
may degrade the image details (edges and textures) and result in visual blur. 

Human visual perception is more sensitive to image edges which consist 
of sharp-changes in the neighboring luminance/chrominance scale. Visual 
perception is essentially adaptive and has variable lenses and focuses ibr different 
visual environments. To protect as much as possible the detailed information, 
while simultaneously removing the noise, modem image processing techniques are 
predominantly based on non-linear methods. Commonly, prior to nonlinear 
filtering, the image edges, as well as perceptually sensitive textures, are detected 
and preserved. The well-known nonlinear filtering approaches include median 
filtering and weighted averaging, etc. Median filtering uses the median 
luminance/chrominance value of the pixels within the neighboring window to 
substitute for the original voxels. This method causes less degradation for the slant 
or square functions, but suppresses the signal impulses whose widths are shorter 
than half of the window width. Thus it can degrade image quality. To pratect the 
edges, weighting averages only smooth neighboring pixels with similar 
luminance/chrominance magnitude. However, a serious shortcoming of these 
methods is that the width of the weighting-window has to be implemented 
adaptively. So the large-scale, complicated calculations are required to generate 
filtering voxels. If the window selected is too wide, more details may be lost. 

Human vision system response has attracted much interest for image 
processing recently [9, 1 8, 23, 40, 43, 49, 54]. Using some newly developed non- 
linear methods (such as perceptually optimal stack filtering [18]) Satisfactory 
restoration images can be obtained. However, this type of nonlinear filter requires 
a thorough knowledge of the ideal image and involves lengthy long-time network 
training, which greatly limits its usefulness. 
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More efficient, human- vision-system-based image processing techniques 
possess the advantages of 1) high de-correlation for convenience of compression 
and filtering; 2) high numerical stability, 3) In addition, in human visual response, 
the filtering algorithm should enhance perceptually sensitive information, while 
simultaneously suppressing the non-perceptual-sensitive components, 4) Finally, 
it can be carried out with real-time processing and is robust. 

The space (time)-scale logarithmic response characteristic of the wavelet 
transform is quite similar to that of the HVS response. Visual perception is 
sensitive to narrow band, low-frequency components, and insensitive to wide- 
band, high frequency components. This feature can be dealt with using the 
constant-Q analysis of wavelet transforms, which possesses fine resolution in the 
low-band regime, and coarse frequency resolution in the high band regime. The 
recently discovered biological mechanism of the human visual system shows that 
both multiorientation and multiresolution are important features of the human 
visual system. There exist cortical neurons which respond specifically to stimuli 
within certain orientations and frequencies. The visual system has the ability to 
separate signals into different frequency ranges. The evidence in 
neurophysiological and psychophysical studies shows that direction-selective 
cortex filtering is much like a 2D-wavelet decomposition representation. The high- 
pass expansion coefficients of a wavelet transform can be regarded as a kind of 
visible difference predictor (VDP). 

The use of wavelets for the task of image restoration and enhancement is 
a relatively new but rapidly emerging approach [3, 26, 27, 29, 38, 40]. Although 
there has long been the view that a non-stationary approach may improve results 
substantially compared to a stationary one, the idea of multiresolution has not been 
a prevalent one. Instead, adaptive restoration techniques have been used to 
examine problems in the spatial domain. Various local measures are employed to 
describe the signal behavior near a pixel. Because local adaptivity is based 
expUcitly on the values of the wavelet coefficients, it is very easy to implement 
and requires a much smaller effort than the conventional deconvolution filter. 
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To enhance the performance of wavelet techniques for signal-noise-ratio 
improvement in signal restoration, it is typical to utilize thresholding techniques 
in multiscale spaces [13, 26, 27, 37, 38]. However, for perceptual image 
processing dependent on the human vision system (HVS), this technique appears 
not to be optimal. As is well known, the HVS model addresses three main 
sensitivity variations, namely, the light (luminance/chrominance) level, spatial 
frequency, and signal content. The visual content of the wavelet coefficients in 
various subblocks (representing different spatial frequency band) is quite different. 
Additionally, the contrast level seriously affects the level of a threshold cutoff. 
Simple multi-channel thresholding methods do not yield perfect perception quaHty 
since they are not fully consistent with HVS response. Furthermore, due to the 
recurrent structure of the subband filters, the coefficient strength in each sub-block 
is also rapidly varying. In addition to these difficulties, the problem of color image 
restoration presents a unique problem in that the multiple color channels are not 
orthogonal Thus, cross-channel correlation must be exploited in order to achieve 
optimal restoration results. A number of approaches have been used to handle the 
color multichaimel image restoration problem [2, 3, 31, 49]. However, the re- 
normalization of the subband response and the HVS has not yet been studied 
thoroughly. 

The present paper deals with these issues and seeks to develop an efficient 
method for noisy color image restoration. We will concentrate on YCrCb channels 
because they are relatively de-correlated and can be processed independently. 
Magnitude normalization (MN) is utilized to adjust the wavelet transform 
coefficients in the different sub-channels. Also, different visual weightings are 
employed in the three luminance-chrominance channels (Y, Cr, Cb) to make the 
wavelet transform coefficients in these channels better match the vision system 
response better. These visual weightings have been used before by Waston to 
generate a perceptual lossless quantization matrix for image compression [49]. For 
non-standard image brightness level (contrast), visual sensitivity normalization 
(VSN) is also needed to fix the cutoff threshold. We refer to these combined three 
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normalization processes as Color Visual Group Normalization (CVGN). 

In applications, recently developed interpolating Lagrange wavelets [41, 
50] are utilized for color image decomposition and reconstructiofl; This kind of 
interpolating wavelet displays a slightly better smoothness and more rapidly time- 
frequency decay than commonly used wavelets. Moreover, the interpolating 
processing enables us to utilize a parallel calculation structure for efficient real- 
time implementation. Most importantly, our multiresolution image processing 
method is extremely robust in not requiring prior knowledge of either an ideal 
image or noise level. 

COLOR MODULATION 

Commonly used color models are of three kinds: 

(1) Computer Graphics Color Space 

CMY: Cyan, Magenta and Yellow 

HLS: Hue, Lightness and Saturation 

HSV (HSB): Hue, Saturation and Value (Brightness) 

RGB: Red, Green and Blue 

(2) TV Broadcast Signal Color Space 

YCrCb: Intensity, Color-red and Color-blue 
YIQ: Luminance, In-Phase and Quadrature 

(3) The CIE (Commission Internationale de Eclairage) Uniform Color Space 

CIEXYZ: Standard primaries X, Y and Z 
CIELab: Luminance, a value and b value 
CIELuv: Luminance, u value and v value 
CIExyY: x-y coordinate and Luminance 

A color vector is usually represented by its three components C=[R, G, B] 
of red, green and blue primary signals (Figure 1), with each signal being 
represented with 8-bit precision (i.e. an integer range of [0, 255]). In color cathode 
ray tube (CRT) monitors and raster graphics devices, the primarily used color 
model is red, green, and blue (RGB). It is also by far the most commonly used 
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model for computer monitors. The Cartesian coordinate system is employed in this 
model. The RGB primaries are additive, such that individual contributions of each 
primary are added for the creation of a new color. The model is based on the tri- 
stimulus theory of vision and is a hardware-oriented model. 

This model can be represented by the unit cube which is defined on R, G, 
and B axes (Figure 2). The line that runs from Black (0,0,0) to White ( 1 , 1 , 1 ) is the 
gray scale line [56]. The values of R, G and B should be the same (R=G=B) in 
order to have an achromatic (colorless) pixel. A full view of a colored RGB cube 
can be seen here. The RGB color model is additive, so that: 

Red + Green= Yellow 

Red + Blue = Magenta (2) 
Green + Blue= Cyan 
Red + Green + Blue = White 
The color cube can also be projected as a hexagon as Figure 3, with the 
lightest point in the middle. This alternative representation is useful for 
understanding the relationship to other proposed color models. For example, it 
helps to visualize the color transformations between several other color models 
(e.g. CMY, HSV, HLS, etc.). With the use of a hexagon, a color RGB cube can be 
represented [56]. 

Let RL, GL, and BL, be values of three color primaries that are equal to (or 
proportional to) the measured luminance of the color primaries. It should be noted 
that these primaries cannot be displayed directly because most monitors exhibit 
a nonlinear relationship between the input value of the color signal and the 
corresponding output luminance. In fact, the nonlinearity for a particularly primary 
can be approximated by a power law relationship: 

Q=C^, C^R, G, B] (3) 
where C is the input value of the primary, Q is the luminance of the primary, and 
/is a value that usually falls between 2 and 3, depending on the monitor used. The 
nonlinearity can also be characterized by more complicated models. In order to 
account for this nonlinearity, a transform on the linear color primaries of the form 
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R^R,'^G=G^'rB=B^^'r^ (4) 
is typically performed as display on the given monitor. The gamma-corrected 
coordinates are also used to characterize many device-independent color spaces. 
One such color space is SMPTE RGB color space, which has been selected as a 
5 television standard. The value of /for this color space is 2.2. All the color images 

used in this work are taken to be in SMPTE gamma-corrected RGB coordinates. 

Since the human vision system (HVS) perceives a color stimulus in terms 
of luminance and chrominance attributes, rather than in terms of R, G, B values, 
we propose to transforai the image to a luminance-chrominance space prior to 
10 performing the quantization. The color representation scheme used in this work 

is the CCIR-601-1 eight-bit digital coding standard, comprising a luminance (Y) 
and chrominance (Cb/Cr) components, with black at luma code 16 and white at 
luma code 235 (instead of the foil 8-bit range of 0 to 255). To this end, we pick the 
YCrCb component color space that is related to the SMPTE RGB space by a 
1 5 simple linear transformation. Assuming that R, G, B occupy the range 0-255, the 

transform is given by 

y=0.2989937?+0.587016G4-0.1139915 

Cr=128+0.713267(/?-y) (5) 
Cfe=128-f0.564334(B-y), 
20 where the Y, Cr, and Cb values have been scaled to the range 0-255. Hence forth, 

3D color vectors will be assumed to be in YCrCb coordinates. Since YCrCb is a 
linear transform of a gamma-corrected RGB space, it is also a linear gamma- 
corrected space: Y is the gamma-corrected luminance component representing 
achromatic colors, the Cr coordinate describes the red-green variation of the color, 
25 and the Cb coordinate describes the yellow-blue variation of the color. The 

conversion to YCrCb is one of the key factors enabling us to achieve high image 
quality. 

Digital grayscale images typically contain values that represent so-called 
gamma-corrected luminance Y. Here Y is a power fonction of luminance, with an 
30 exponent of around 1/2.3. Three gamma-corrected numbers, red, green, and blue 
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components (RGB), are used to represent each pixel in the case of color images. 
In the case of a gray image, each pixel is represented by a single brightness. In this 
treatment, each pixel is transformed from the original color representation (for 
example RGB) to a color representation that consists of one brightness signal and 
two color signals (such as the well-known TV broadcast color space YCbCr). 



Table 1 . Display resolution comparison 



Display 


Resolution 


Distance 


DVR 




(pixels/inch) 


(inches) 


(pixels/degree) 


Computer Display 


72 


12 


15.1 


Low Quality Printing 


300 


12 


.62.8- 


High Quality Printing 


1200 


12 


251.4 


HDTV 


48 


72 


60.3 



VISUAL SENSITIVITY OF WAVELET COEFFICIENTS 

The visibility of wavelet transform coefficients will depend upon the display 
visual resolution [49] in pixel/degree. Given a viewing distance Fin inches and a display 
resolution d in pixel/inch, the effective display visual resolution (DVR) R in pixel/degree 
of visual angle is 

R=rfKtan(;z/180)«rfr/57.3 (6) 
A useful measurement is that the visual resolution is the viewing distance in pixels {dV) 
divided by 57.3. Table 1 provides some illustrative examples. For example, the HDTV 
assumes 1 152 active lines at a viewing distance of 3 picture heights [49]!*'. 

On each decomposition level, wavelet coefficients are divided into four 
subblocks, LL, HL, LH, and HH. The detailed subblocks HL, LH and HH represent three 
different decomposition orientations, horizontal, vertical, and diagonal, respectively. As 
the layer increases, the bandwidth of the equivalent subband filters decreases by a factor 
of two, and thus the frequency resolution doubles. Correspondingly, the space (time) 
resolution (display resolution) decreases by a factor of two. A subblock of wavelet 
coefficients corresponds to a spatial fi-equency band. For a display resolution of R 
pixel/degree, the spatial fi-equency / of level j is 

f-T^R (7) 

As discussed in section I, the display gamma is 2.3. The R, G, B true color image- 
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is modulated into Y, Cr, Cb channels. The model thresholds for three-color channels are 
different from each other. For example, the just-noticeable quantization threshold of Y 
is generally about a factor of two below that of Cr, which is in turn about a factor of two 
below the Cb curve at each spatial frequency. Note that this difference usually declines 
at higher spatial frequency [49], The Cb curve is somewhat broader than Y or Cr. This 
broadening is likely due to the intrusion of a luminance-detecting channel at high 
frequencies and high contrasts. Because the Cb color axis is not orthogonal to the human 
luminance axis, the Cb colors do have a luminance component. 

The contrast sensitivity declines when the spatial frequency increases, whereas, 
the size of stimuli decreases. This mode is adapted to construct the '^perceptual lossless" 
response magnitude for normalizing in subblock according to the visual response. 

VISUAL GROUP NORMALIZATION 

The main objective of wavelet signal filtering is to preserve important signal 
components, and efficiently reduce noisy components. For perceptual images, it is most 
important to protect the signal components (always represented by the 
luminance/chrominance levels) which are sensitive to human eyes. To achieve this goal, 
we utilize the magnitudes of the filter response and the human vision response. 

Magnitude Normalization 

Wavelet coefficients can be regarded as the output of the signal passing through 
equivalent decomposition filters (EDF). The responses of the EDF are the combination 
of several recurrent subband filters at different stages [19, 28, 34, 37, 385-40, 41]. The 
EDF ampHtudes of various sub-blocks differ greatly. Thus, the magnitude of the 
decomposition coefficients in each of the sub-blocks cannot exactly reproduce the actual 
strength of the signal components. To adjust the magnitude of the response in each block, 
the decomposition coefficients are re-scaled with respect to a common magnitude 
standard. Thus EDF coefficients, CJJc\ in block m should be multiplied with a 
magnitude scaling factor, to obtain an adjusted magnitude representation. We choose 
this factor to be the reciprocal of the maximum magnitude of the frequency response of 
the equivalent wavelet decomposition filter on node ij,m) 
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This idea was recently extended to Group Normalization (GN) of wavelet packets for 
signal processing [37, 38, 40, 41] and was presented to lead the optimal performance. 
Perceptual Lossless Normalization . ^ 

Because an image can be regarded as a signal source based on the human visual 
system, using a just-noticeable distortion profile we can efficiently remove the visual 
redundancy fi-om decomposition coefficients and normalize them with respect to the 
standard of perceptual importance. A mathematical model for perception efficiency, 
based on the amplitude nonlinearity in different frequency bands, has been presented by 
Watson, et al [49], which can be used to construct the "perceptual lossless" response 
magnitude Yj^^ for normalizing visual response in different luminance/chrominance 
spaces. We extend the definition to luminance/chrominance mode according-to 



where defines the minimum threshold, is a constant, is the Display Visual 
Resolution (D VR),^ ^ is the spatial frequency, and ^ is the directional response factor, 
in each luminance/chrominance channel v. 

The parameters J^^,, and rf^^^^represent the thresholds for orientations LL and HL 
as frequency shifts relative to threshold for orientations LH and HL. From the nature of 
dyadic wavelets, the orientation LL possesses a spectrum that is approximately a factor 
of two lower in spatial frequency than orientation LH or HL. This would suggest a factor 
of ^u.v=2. However, in a vision system response mechanism, at orientation-LL the signal 
energy is spread over all orientations, which implies less visual efficiency than when the 
energy is concentrated over a narrow range, as is the case for the spectra of orientation 
HL or LH. Thus, the threshold should be increased, which can be realized by a slight 
reduction in^. 

The final value for is therefore less than 2. 

For orientation HH, similar effects exist. First, the Cartesian splitting of the 

spectrum makes the spatial frequency of orientation HH about V2 higher than that of the 

orientations HL, and LH. Thus df^^^ can be taken as d^„f=l'^^. As we mentioned before, 
the spectrum along orientation HH is spread over two orthogonal orientations (45° and 
135°), which should result in a log threshold increase of about 2*'"* (a shift of 2''''*) or a 




(9) 
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total prediction of rf//^^=2"^''*=0.59. Finally, the well-known oblique effect [49], will cause 
a final small amount of threshold elevation. 

The chromatic channels Cb and Cr are usually each down-sampled by a factor of 
two in both horizontal and vertical directions, because the sensitivity of human vision to 
5 chromatic variation is weaker than it is to luminance variation. In the calculation of the 

lossless quantization matrix, we should in this case make some correction. For example, 
if the display visual resolution is known, as the chroma that is down-sampled by two in 
each dimension, the corrected value will be half of that. This corrected value is used to 
adjust the perceptual lossless normalization in Cr and Cb channels. 

10 

Visual Sensitivity Normalization 

Visual sensitivity in each luminance/chrominance channel v is defined as the 
inverse of the contrast required to produce a threshold response [9], 

S=VC, (10) 
15 where Q is generally referred to simply as the threshold. The Michelson definition of 

contrast, 

Q~(^inar.v"-^mcart,v)/^iiiea/i.v (11) 

is used, where ^ and L„^^^ refer to the maximum and mean luminances of the 
waveform in luminance/chrominance channel v. Sensitivity can be thought of as a 

20 gain, although various nonlinearities of the visual system require caution in the use of 

this analogy. The variations in sensitivity as a fimction of light level are primarily due 
to the Ught-adaptive properties of the retina and are referred to as the amplitude 
nonlinearity of the HVS. The variations as a fimction of spatial fi-equency are due to 
the optics of the eye combined with the neural circuitry; these combined effects are 

25 referred to as the contrast sensitivity function (CSF). Finally, the variations in 

sensitivity as a fimction of signal content referred to as masking^ are due to the post- 
receptoral neural circuitry. 

Combining the perceptual lossless normalization, the visual sensitivity 
normalization and the magnitude normalized factor Aj^ we obtain the perceptual 

30 lossless quantization matrix Qj„ ^ 

Qj.m,v'ZCvYj^„^yAj^„ (12) 

This treatment provides a simple human- vision-based threshold technique for the. 
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restoration of the most important perceptual information in an image. We call the 
combination of the above-mentioned three normalizations Color Visual Group 
Normalization (CVGN) of wavelet transform coefficients. 

RESULTS OF EXAMPLE APPLICATIONS 

Our study suggests that the analyzing functions presented in this paper can 
improve the visualization of features of importance to color images. As discussed earlier, 
the color channel application of the Color Visual Group Normalization (VGN) technique 
is designed to improve the performance of our interpolating Lagrange wavelet transform 
[41] in image processing. Utilizing Visual Group Normalization, the raw magnitudes of 
the transform coefficients in different luminance and chrominance channels can be 
normalized to represent exactly the visual perceptual strength of signal components in 
each subband. Moreover, the efficient non-linear filtering method — softer logic masking 
(SLM) technique [38], provides robust edge-preservation for image restoration, and 
removes the haziness encountered with the commonly used hard-logic filtering 
techniques. 

In our study, the original benchmark color photo of Lena was cropped to a matrix 
size of 5 1 2x5 1 2. Another color picture is digitized by Kodak digital camera with matrix 
size 512x768. The original images possess clear edges, strong contrast and brightness. 
Figure 4(a) is the typical noisy Lena image degraded by adding Gaussian random noise. 
A simple low-pass filter smoothes out the noise but also degrades the image resolution, 
while a simple high-pass filter can enhance the texture edges but will also cause 
additional distortion. We choose 2D half-band Lagrange wavelets as the multiresolution 
analysis tools for image processing. 

The median filtering (with a 3x3 window) result of Lena is shown in Figure 4(b), 
which is edge-blurred with low visual quality. The speckled noise has been changed into 
bumps. It is evident that our Color Visual Group Normalization technique yields better 
contrast and edge-preservation results and provides a more natural color description of 
this image (Figure 4(c)). 

For a second example, the noisy picture, result of median filtering and our result 
are shown in Figure 5(a), 5(b) and 5(c), respectively. Our previous conclusion pertain, 
showing that the CVGN technique is also suitable for image processing of pictures 
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having a different height-width ratio. 
CONCLUSIONS 

Blind image restoration and de-noising are very difficult tasks. Because of the 
5 complex space-frequency distribution and statistical feature of images, there is almost no 

definite feature discrimination between image and noise background. Even if prior 
knowledge of the noise is available, a perfect extraction is theoretically impossible. The 
key problem in signal processing is how to restore images "naturally" from a noisy 
background or from an encoded quantimtion space. For a de-noising scheme based on 
10 statistical properties, an adaptive signal-dependent transform is needed to generate a 

concentrated representation of signal components. This is based on the assumption that 
noise or clutter components in the signal subspace are relatively scattered with lower 
magnitudes. 

The logrithmic frequency band distribution of our wavelet decomposition is 

1 5 matched well with the characteristics of human vision responses to obtain compact signal 

representation. Both multiorientation and multiresolution are known features of the 
human visual system. There exist cortical neurons that respond specifically to stimuli 
within certain orientations and frequencies. The 2-D wavelet decomposition mimics the 
cortex filtering of the HVS. Visual group normalization is proposed to nonmalize the 

20 frequency response so that the wavelet coefficients can exactly represent the perceptual 

strength of signal components. The color chaimel based VGN technique results in a 
thresholding method that is efficient for visual feature extraction. 

We also employ a modified version of a well-known threshold method (termed 
the Perceptual Softer Logic Masking (PSLM) technique [38, 40]) for image restoration, 

25 which dealing with extremely noisy backgrounds. This technique better preserves the 

important visual edges and contrast transition portions of an image than does the 
traditional method, and it is readily adaptable to human vision. The smooth transition 
around the cut-off region of the filter can efficiently remove the Gibbs' oscillation in the 
signal reconstruction, and decrease the appearance of ringing and aliasing: A symmetric 

30 extension decomposition [6] is utilized to remove non-continuous boundary effects. The 

interpolation algorithm made possible by use of an interpolating wavelet transform [12, 
14, 15, 16, 25, 35, 36, 41] enables us to utilize a parallel computational strategy that is 
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convenient for real-time implementation. 

To some extent, color visual group normalization provides a measurement for 
handling raw filter coefficients in a wavelet transform. The hierarchical filtering of 
Softer-logic Masking can protect more signal components than a simple single layer 
nonlinear method. The concept of Visual Lossless Quantization (VLQ) presented in [49] 
can lead to a potential breakthrough compared to the traditional Shannon rate-distortion 
theory in information processing. We have compared our approach with the commonly 
used median filtering method for de-noising. The results show that the proposed method 
is robust and provides the best quality for color image filtering of which we are aware. 
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MAMMOGRAM ENHANCEMENT USING GENERALIZED STNC WAVELETS 
INTRODUCTION 

Interpolating Distributed Approximating Functionals (DAFs), designed as a set 
of envelop-modulated interpolants, are generated by Gaussian-modulated Sine, Hermit, 
or Lagrange functionals [15, 31, 40, 41]. Such DAFs are smooth and decaying in both 
time and frequency representations and have been used for numerically solving various 
linear and nonlinear partial differential equations with extremely high accuracy and 
computational efficiency. Examples include DAF-simulations of 3-D reactive quantum 
scattering, the Kuramoto-Sivashinsky equations describing flow pattern dynamics for a 
circular domain, the sine-Gordon equation near homoclinic orbits, and ■a*2-D Navier- 
Stokes equation with non-periodic boundary conditions. Because the interpolating core 
shell of the fundamental DAF is the interpolating Sine functional (the ideal low-pass 
filter), it is naturally useful to construct interpolating wavelets from these DAFs for use 
in signal processing. 

The theory of interpolating wavelets has attracted much attention recently [1,8, 
10, 11, 12, 19, 20, 26, 27, 30, 31, and 32]. It possesses the attractive characteristic that 
the wavelet coefficients are obtained from linear combinations of discrete samples rather 
than from traditional inner product integrals. Mathematically, various interpolating 
wavelets can be formulated in a biorthogonal setting and can be regarded as an extension 
of the auto-correlation shell wavelet analysis [26], and halfband filters [1]. Harten has 
described a kind of piecewise polynomial method for biorthogonal interpolating wavelet 
construction [12]. Swelden independently develops this method as the well-known 
interpolating "lifting scheme" theory [32], which can be regarded as a special case of the 
Neville filters [19]. Unlike the previous method for constructing the biorthogonal 
wavelets, which attempts to explicitly solve the relevant coupled algebraic equations 
[10], the lifting scheme enables one to construct a custom-designed biorthogonal wavelet 
transform just assuming a single low-pass filter (such as DAF filter or scaling fimction) 
without iterations. 

Generally speaking, the lifting-interpolating wavelet theory is closely related to 
the finite element technique in the numerical solution of partial differential equations, the 
subdivision scheme for interpolation and approximation, multi-grid generation and 
surface fitting techniques. The most attractive feature of the approach is that discrete 
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samplings are made identical to wavelet multiresolution analysis. Without any pre- 
conditioning or post-conditioning processes for accurate wavelet analysis, the 
interpolating wavelet coefficients can be implemented using a parallel computational 
scheme. 

In this paper, the Sinc-DAF is employed to construct new biorthogonal 
interpolating wavelets— DAF wavelets and associated DAF-filters specifically for use 
in mammogram decomposition. Two kinds of different biorthogonal interpolating DAF 
wavelets (B -spline-enveloped Sine wavelets and Gaussian-enveloped DAF wavelets), as 
the examples of the generalized interpolating Sine wavelets, have been discussed 
thoroughly. Because the finite length cutoff implementation of the Sine (ideal low-pass) 
filter causes the Gibbs oscillations, the key idea for the DAF wavelet construction is to 
introduce bell-shaped weighted windows to improve the characteristics of localization 
in time-fi-equency plane, and to ensure perfectly smooth and rapid decay. 

To obtain excellent reconstruction quality of a digital mammogram, human visual 
sensitivity is utilized to construct the visual group normalization (VGN) technique, which 
is used to re-scale the wavelet decomposition coefficients for perceptual adapted 
reconstruction according to human perception. Softer Logic Masking (SLM) is an 
adjusted de-noising technique [29], derived to improve the filtering performance of 
Donoho's Soft Threshold method [9]. The SLM technique more efficiently preserves 
important information and edge transition in a manner particularly suited to human visual 
perception. A nonlinear contrast stretch and enhancement fimctional is easily reaflized for 
wavelet multiscale gradient transformation and feature-sensitive image reconstruction, 
which enables us to obtain accurate space-localization of the important features of the 
mammogram. The combined techniques can greatly improve the visualization of low- 
contrast components of a manmiogram, which is important for diagnosis. Additionally, 
interpolating DAF-wavelet image processing can be implemented as an interpolation 
method. The coefficient calculations therefore only involve simple add/multiply 
operations. This is extremely efficient for fast implementation. 

As an example application of Sinc-DAF wavelets,- we select mammogram image 
processing, de-noising and enhancement, because of its huge data size, complicated 
space-firequency distribution and complex perceptu^ dependent characteristics. 
Perceptual signal processing has the potential of overcoming the limits of the traditional 
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Shannon Rate-Distortion (R-D) theory for perception-dependent information, such as 
images and acoustic signals. Previously, Ramchandran, Vetterli, Xiong, Herley, Asai, and 
Orchard have utilized a rate-distortion compromise for image. ; -compression 
implementation [14, 23, 24, and 42]. Our recently derived Visual Group Normalization 
(V GN) technique [3 1 ] can likely be used with the rate-distortion compromise to generate 
a so-called Visual Rate-Distortion (VR-D) theory to improve image processing further. 



GENERALIZED SINC WAVELETS 

Sine Wavelets 
The ;rband-limited Sine function, 

^x)=sin( 7d:)I{ 7Dc)Cr ( 1 ) 

in Paley-Wienex space, constructs the interpolating functionals. Every band-limited 
function fdL\R) can be reconstructed by the equation 

^<'' = ?^<"^!t^ (2) 
where the related wavelet function — Sinclet is defined as (see Figurel) 

, ^ sin Jt{2x - 1 ) - sin ;r(x - 1 / 2) . . 

''W ^(^rrTTi) (3) 

The scaling Sine function is the well-known ideal low-pass filter which possesses 
the filter response 



0. 7r/2^<D\^7r 



Its impulse response can be described as 

^sm{7m)l{7i) (5) 
The so-called half-band filter only possesses non-zero impulses at odd integer 

samples, /r(2^1), while at even integers, A[2fc]=0, except for ^7=0. 

However, this ideal low-pass filter can not be implemented physically. Because 

the digital filter is an IIR (infinite impulse response) solution, its digital cutofTFIR (finite 
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impulse response) implementation will introduce Gibbs phenomenon (overshot effect) 
in Fourier space, which degrades the frequency resolution. 

The explicit compactly supported Sine scaling function and wayetet, as well as 
their biorthogonal dual scaling function and wavelet, are shown in Figure 3 . We find that 
5 the cutoff Sine has decreased regularity, which is manifested by a fractal-like behavior, 

which implies poor time localization. 

B-Spline Sine Wavelets 

Because the ideal low-pass Sine wavelet can not be implemented "ideally" by FIR 
10 (finite impulse response) filters, a windowed weighting technique is introduced here to 

eliminate the cutoff singularity, and improve the time-frequency localization of the Sine 
wavelets. 

First, we define a symmetric Sine interpolating functional shell as 
15 />W=^1=^ (6) 

Multiplying by a smooth window, which vanishes gradually at the exact zeros of 
the Sine functional, will lead to more regular interpolating wavelets and equivalent 
subband filters (as shown in Figure 4 and 5), 
20 For example, we select a well-defined B-spIine function as the weight window. 

Then the scaling function (mother wavelet) can be defined as an interpolating B-spline 
Sine fimctional (BSF) 

p^jxirj) smimfl) (7) 
25 ^'^(0) ^ 

where is the B-spline order, 7 is the scaling factor to control the window width. To 
ensure coincidence of the zeroes of the B-spline and the Sine shell, we set 

2M+l=7x(AM-l)/2 (8) 
To ensure the interpolation condition, h{2k)-0y A^O, it is easy to show thatwhen the B- 
30 spline order A^=4it+1 , 7 can be any odd integer (2^+1 ). If is even integer, then 7 can 

only be 2. When N-Ak- 1 , we can not construct the interpolating shell using the definition 
above. The admission condition can be expressed as 
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(9) 



From the interpolation relation 

[LJk = 0 



, keZ 



(10) 



and the self-induced two-scale relation 
it is easy to verify that 

h{kyU^2)ll, k=^2M+U 2M-\ (12) 



GAUSSIAN-SINC DAF WAVELETS 
15 We can also select a class of distributed approximating functionals, e.g., the 

Gaussian-Sinc DAF (GSDAF) as our basic scaling function to construct interpolating 
scalings, 

^W^(,)—_ ^^^^ 

20 

where Wj[x) is a window function which is selected as a Gaussian, 

25 Because it satisfies the minimum frame bound condition in quantum physics, it will 

improve the time-frequency resolution of the Windowed-Sinc wavelet. Here is a 
window width parameter, and P(x) is the Sine interpolation kernel. The DAF scaling 
function has been successfully introduced as an efficient and powerful grid method for 
quantum dynamical propagations [40]. The Gaussian window in our DAF-wavelets 

30 efficiently smoothes out the Gibbs oscillations, which plague most conventional y/avelet 

bases. The following equation shows the connection between the B-spline function and 
the Gaussian window [34]: 
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(15) 



5 cr = 7V(^+l)/12 (16) 

the Gaussian Sine wavelets generated by the lifting scheme similar to the B-spline Sine 
wavelets. Usually, the Gaussian Sine DAF displays a slightly better smoothness and rapid 
decay than the B-spline Lagrange wavelets. 
10 If we select more sophisticated window shapes, the Sine wavelets can be 

generalized fiirther. We shall call these extensions Bell-windowed Sine wavelets. The 
available choice can be the different kinds of the popularly-used DFT (discrete Fourier 
transform) windows, such as Bartlett, Hanning, Hamming, Blackman, Chebechev, and 
Besel windows. 

15 

VISUAL GROUP NORMALIZATION 

Filterbank Magnitude Normalization 

On each decomposition level, 2-D wavelet coefficients are divided into four sub- 
blocks, LL, HL, LH, and HH. As usual, L and H represent the low-pass and high-pass 

20 subband filtering results, respectively. For example, HL means the signal passes the 

horizontal high-pass filter first and passes the vertical low-pass filter. Obviously, the 
detailed sub-blocks HL, LH and HH represent the multiscale difference operation in three 
different decomposition orientations, horizontal, vertical, and diagonal, respectively. At 
each level of analysis, the bandwidth of the equivalent subband fihers decreases by a 

25 factor of two. 

Wavelet coefficients can be regarded as the output of the signal passing through 
equivalent decomposition filters (EDF). The responses of the EDF are the combination 
of several recurrent subband filters at different stages [19, 28, 34, 37, 38, 40, 41]. The 
EDF amplitudes of various sub-blocks differ greatly. Thus, the magnitude of the 

3 0 decomposition coefficients in each of the sub-blocks cannot exactly reproduce the actual 

strength of the signal components. To adjust the magnitude of the response in each block, 
the decomposition coefficients are re-scaled with respect to a common magnitude- 
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standard. Thus EDF coefficients, CjJJcX in block (/» should be multiplied with a 
magnitude scaling factor, Xj^, to obtain an adjusted magnitude representation. Here j 
represents the decomposition layer, and m denotes the different orientation block (LL, 
LH,HLorHH). 

We choose the normalizing factor to be the reciprocal of the maximum magnitude 
of the frequency response of the equivalent wavelet decomposition filter on node 



10 Thus, the magnitude normalized coefficients, NCj JJc), are defined as 

^^jM-^j,.C,M^ (18) 

This idea was recently extended to Group Normalization (GN) of wavelet packets for 
15 signal processing [37, 38, 40, 41] and was shown to yield the optimal performance. 

Filterband magnitude normalization (FMN) unifies the coefficient strength in 
each of the subblocks. However, an image is a perceptual signal source. Coefficients with 
equal magnitude (after FMN) in different firequency channels result in greatly different 
visual gain (sensitivity) for human eyes. Additional adjustments of the wavelet 
20 coefficients are required for visual image processing. 

Perceptual Lossless Normalization 

The reconstruction visibility of wavelet transform coefficients wilhxiepend upon 
the display visual resolution [39] in pixel/degree. Given a viewing distance Fin inches 
and a display resolution d in pixel/inch, the effective display visual resolution (D VR) R 
25 in pixel/degree of visual angle is 

R^dVtan{nim)^dVI513 (1 9) 

The visual resolution is the viewing distance in pixels {dV) divided by 57,3. 

As mentioned above, when the decomposition layer increases, the bandwidth of 
the equivalent subband filters decreases by a factor of two; Thus the fi-equeircy resolution 
30 doubles. Correspondingly, the space (time) resolution (display resolution) decreases by 

a factor of two. A sub-block of wavelet coefficients corresponds to a spatial firequency 
band. For a display resolution of R pixel/degree, the spatial frequency / of level j is 
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(20) 



For a Y-channel gray-scale mammogram image, the just-noticeable quantization 
threshold of Y is generally different at each spatial frequency. The contrast sensitivity 
declines when the spatial frequency increases (whereas, the size of the stimuli decreases). 
This fact is used to construct the "perceptual lossless" response magnitude for 
normalization in subblock (j,m) according to the visual response. 

For images based on the human vision system (HVS), using a just-noticeable 
distortion profile, we can efficiently remove the visual redundancy from decomposition 
coefficients and normalize them with respect to the standard of perceptual importance. 
A simple mathematical model for perception efficiency, based on the amplitude 
nonlinearity in different frequency bands, has been presented in [39], which can be used 
to constmct the "perceptual lossless" response magnitude Yj^ for normalizing visual 
response in different luminance/chrominance spaces. We extend the definition to 
Itmiinance/chrominance modes according to 



where a defines the minimxun threshold, is a constant, R is the Display Visual 
Resolution (DVR),^^ is the spatial frequency, and d„ is the directional response factor 
for each subblock. 

The parameters d^, and rf^^represent the thresholds for orientations LL and HL 
as frequency shifts relative to the threshold for orientations LH and HL. From the nature 
of dyadic wavelets, the orientation LL possesses a spectrum that is approximately a factor 
of two lower in spatial frequency than orientation LH or HL. This would suggest a factor 
of ^Lz,~2. However, in a vision system response mechanism, at orientation LL the signal 
energy is spread over all matrix elements, which implies less visual efficiency than when 
the energy is concentrated over a narrow range, as is the case for the spectra of 
orientation HL or LH. Thus, the threshold should be increased, which can be achieved 
by a slight reduction in /q. The final value for c/^ is therefore less than 2. ■ 

For orientation HH, similar effects exist. First, the Cartesian splitting of the 

spectrum makes the spatial frequency of orientation HH about V2 higher than that of 
the orientations HL, and LH. Thus d„„C2n be taken as d„fj=^2'^^. As we mentioned before, 




(21) 
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the spectrum along orientation HH is spread over two orthogonal orientations (45° and 
135°), which should result in a log threshold increase of about 2""* (a shift of 2"''^) or a 
total threshold factor of rf^;/=2'^'*=0.59. Finally, the well-known oblique, effect [39], will 
cause a small amount of threshold elevation. 

Visual Sensitivity Normalization 

Visual sensitivity is defined as the inverse of the contrast required to produce a 
threshold response [9], 

5=1/C (22) 
where C is generally referred to simply as the threshold. The Michelson definition of 
contrast, 

CHKa.'L^^nyL^ean (23) 

is used, where L^^ and refer to the maximum and mean luminances of the 
waveform in a luminance channel. Sensitivity can be thought of as a gain, although 
various nonlinearities of the visual system require caution in the use of this analogy. The 
variations in sensitivity as a function of light level are primarily due to the light-adaptive 
properties of the retina and are referred to as the amplitude nonlinearity of the HVS, The 
variations as a fimction of spatial fi-equency are due to the optics of the eye combined 
with the neural circuitry; these combined effects are referred to as the contrast sensitivity 
function (CSF). Finally, the variations in sensitivity as a fimction of sigpal content, 
referred to as masking, are due to the post-receptoral neural circuitry. 

Combining the perceptual lossless normalization, the visual- sensitivity 
normalization and the magnitude normalized factor^y^, we obtain the perceptual lossless 
quantization matrix „ 

Q,„^2CYjJA^^ (24) 
This treatment provides a simple human-vision-based noraialization technique for the 
restoration of the most important perceptual information in a mammogram image. We 
call the combination of the above-mentioned three normalizations the Visual Group 
Normalization (VGN) of wavelet transform coefficients. 

IMAGE PROCESSING TECHNIQUES 
Softer Logic Masking 
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The main objective of wavelet signal filtering is to preserve important signal 
components, and efficiently reduce noise components. 

After visual group normalization processing, coefficients have been normalized 
according to human visual sensitivity. An additional filtering algorithm is required for 
de-noising and removing perceptual redundancy. Hard logic masking is the commonly 
used nonlinear processing technique. It is similar to a bias estimated dead-zone limiter, 
Jain [14] has shown that a non-linear dead-zone limiter can improve the SNR for weak 
signal detection. It can be expressed as 

v{y)-^gM[\y\-sl^^\<p^\ (25) 

where ^^is a threshold value, and 3/ is the measurable value of the coefficient. Donoho 
shows that the fi= 1 case of the above expression is a near optimal estimator for adaptive 
NMR data smoothing and de-noising [11]. Independently, we utilized the hard logic 
masking to efficiently extract a target from formidable background noise in a previous 
work [26, 27, 28], 

Various threshold cutoffs of multi-band expansion coefficients in hard logic 
masking methods are very similar to the cutoff of an FFT expansion. Thus, the Gibbs 
oscillations associated the FFT will also occur in the wavelet transform using a hard logic 
masking. Although hard logic masking methods with appropriate threshold values do not 
seriously change the magnitude of a signal after reconstruction, they can cause 
considerable edge distortions in a signal due to the interference of additional high 
frequency components induced by the cutoff. The higher the threshold value, the larger 
the Gibbs oscillations will be. 

Edges are especially important for feature preservation and precise localization 
for images and biomedical signals. We here present a Softer Logic Masking (SLM) 
method. In our SLM approach, a smooth transition band near each masking threshold is 
introduced so that any decomposition coefficients, which are smaller than the threshold 
value, will be reduced gradually to zero rather than h6 set to zero. This treatment 
efficiently suppresses Gibbs oscillations and preserves signal edges, and consequently 
improves the quality of the reconstructed signal. Our SLM method can be expressed as 
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(26) 



where the are the decomposition coefficients to be retained in thexeconstruction 



and the quantity NCj\m(k) is defined as 



(27) 



10 



15 



The softer logic mapping, 5OFr:[0,l]->[0,l], is a non-linear monotonically increasing 
sigmoid functional. A comparison of the hard and softer logic masking functionals is 
depicted in Figure 7. 

The softer logic functional can also be taken as the alternated form 



(28) 



where ^ is a normalized adaptive threshold. For an unknown noise level, an 
approximation to ^is given as 



^ = r.ppcA^l2\ogNiN . ^^^^ 



20 



where ais a scaling factor and can be chosen as ^7^1/1.349. The quantity y^^^^^ is an 
upper frame boundary of the wavelet packet transform, i.e., the upper boundary singular 
value of the wavelet transform decomposition matrix. Using arguments similar to those 
given by Donoho [1 1], one can show that the above Softer Logic Masking reconstruction 
is a near optimal approximation in the min-max error sense. 
25 Device Adapted Enhancement 

The basic idea is to use gradient operators to shape flat image data so that desired 
portion of the image is projected onto a screen. 

Using visual normalization and softer-logic thresholding, one can efficiently 
remove the visual redundancy and noisy components from the decomposition 
30 coefficients. 

For grayscale image contrast stretching, the objective is to improve the perception . 
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capability for image components which the human visual system is initially insensitive 
but is important for diagnosis. In other words, mammogram enhancement increases the 
cancer detection probability and precision. We first appropriately normalize the 
decomposition coefficients according to the length scale of the display device so that they 
fall within the interval of the device fi-ame. Assume that the image coefficients have 
already been properly scaled by visual group normalization so that the amplitude value 
NCj Jk) falls into the dynamic range of the display device: 

d^^<NCjJk)<d^^ (30) 
Without loss of generality, we consider the normalized gradient magnitude, 

U,Jk)=NCjJk)l(d^-d^:) (31) 

Mallat and Zhong realized that wavelet multiresolution analysis provides a natural 
characterization for multiscale image edges, and these can be easily extracted by various 
differentiations [15]. Their idea was extended by Laine et al [7] to develop directional 
edge parameters based on a subspace energy measurement. An enhancement scheme 
based on complex Daubechies wavelets was proposed by Gagnon et al. [9] . These authors 
made use of the difference between the real and imaginary parts of the wavelet 
coefficients. One way or another, adjusted wavelet transforms should be designed to 
achieve desired edge enhancement. 

Our starting point is taken to be the magnitude normalized or visual group 
normalized wavelet subband coefficients NC^ Jk) [10, 12]. We define an enhancement 
fimctional£j„ 

j,m 

(32) 

where A is the Laplacian and 'l<aj^, J3j^<\, The coefficients can be easily 

chosen so that desired image features are emphasized. In particular we can emphasize an 
image edge of selected grain size. We note that a slight modification of Uj ^ and can 
result in orientation selected image enhancement. A detailed discussion of this matter will 
be presented elsewhere. An overall re-normalization is conducted after image 
reconstruction to preserve the energy of the original image. We call this procedure 
enhancement normalization. 

Contrast stretching is an old but quite efficient method for feature selective image 
display. Nonlinear stretching has been used by many authors [3, 7, and 16]. Lu and 
coworkers [ 1 6] have recently designed a hyperbolic fimction 
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g//:)=[tanh(a^-6)+tanh(6)]/[tanh(a-6)+tanh(6)] (33) 
for wavelet multiscale gradient transformation. Their method works well for lunar 
images. The basic idea is to use gradient operators to shape a flat image s(T that desired 
portion of the image is projected into a screen window. 

5 

ENHANCEMENT RESULT 

To test our new approaches, digital breast mammogram images are employed. 
Mammograms are complex in appearance and signs of early disease are often small 
and/or subtle. Digital mammogram image enhancement is particularly important for 

1 0 aiding radiologists in the development of an automatically detecting expert system. The 

original image is coded at 768x800 size and a 200-micron pixel edge as shown-in Figure 
8(a). As shown in Figure 8(b), there is a significant improvement in both edge 
representation and image contrast resulting from DAF-wavelets combined with our 
Visual Group Normalization (VGN) and non-linear enhancement techniques. In 

15 particular, the domain and intemal structure of higher-density cancer tissues are clearly 

displayed. The results are characterized by high-quahty image enhancement and good 
signal averaging over homogeneous regions with minimal resolution degradation of 
image details. 

20 CONCLUSION 

The newly developed DAF-wavelet image processing and enhancement 
techniques can improve present and future imaging performance for earlier-detection of 
cancer and malignant tumors. It improves the spatio-temporal resolution of biomedical 
images and enhances the visualization of the perceptually less-sensitive components that 
25 are very important for diagnosis, as well as reduces the distortion and blur. 

Different image processing techniques (distortion suppression, enhancement, and 
edge sharpening) can be integrated in a single step process or can be alternatively applied 
by different coefficient selection. Our biomedical image processing and computer vision 
research should prove important in meeting hospital needs in image enhancement, image 
30 analysis and expert diagnosis. 

The method presented can be applied to various types of medical images. These 
include various X-ray images, Mammography, Magnetic Resonance Imaging (MRI), - 
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Supersonic imaging, etc. Enhanced imaging of internal organs and/or other parts of the 
human body, for the detection of cancer and other diseases, is of great importance. It has 
the potential for earlier, cost-effective diagnosis and management of disease, can provide 
improved visualization of normal versus diseased tissue, and enhance the study of drug 
5 diffusion and cellular uptake in the bram. 

The method dramatically improves the image quality (in terms of signal to noise 
improvements and/or contrast differentiation) by de-noising and enhancement, as 
demonstratedly the digital mammogram example. 
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DUAL PROPAGATION INVERSION OF FOURIER AND LAPLACE SIGNALS 
INTRODUCTION 

In many physical and chemical phenomena, two functions are related by an integral 
equation of the form 



where Ar(x^') is some sort of "Transform Kernel", and jc, x* may be conjugate variables 
(depending on the transform under consideration). For experimental or computational 
studies, the function C{x) is typically known only on a finite, discrete set of points. 
Extractingy(xO fro^i measured or calculated values of C(x) is referred to: as "inversion". 
Famous examples of this inversion problem are the Fourier and Laplace transforms. 
Although there are welHoiown, exact mathematical procedures for these inversions, one 
generally faces serious difficulties in numerically determining the function f(x*) from 
C(x)-values known at a finite number of discrete points. In the Laplace transform, for 
example, the inversion may be unstable due to amplification of noise or other errors in 
the C(Xi) data [1]. Inverse Fourier transforms can also suffer from slow convergence of 
the integration, requiring a large range of samples of the C(jcj), so that one is able to 
obtain only low resolution spectra when a sufficiently long duration time signal is not 
available [2,3]. 

A convenient procedure for numerically inverting such an integral equation is 
disclosed, with the focus here being primarily on real-time spectra. We shall demonstrate 
the method using a simple Fourier transfomiation as an example, but the procedure may 
be useful for other inversion problems. Our method, termed the "dual propagation 
inversion"(DPI) method, makes use of the distributed approximating functional 
(DAF)free propagator [4] to carry out the inversion, but other numerical free propagator 
techniques (such as the Fast Fourier method [5,6]) may also be used. The Fourier 
transform is naturally invoked whenever one tries to obtain information in the 
frequency-domain from a time-domain signal. In geineral, for real - spectra, the 
time-domain and frequency-domain spectra are related by 




(1) 




(2) 
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and 



(3) 



Typical examples of the sorts of problems of interest are the computation of absorption 
spectra or Raman spectra [7] from the autocorrelation function and the extraction of 
vibrational frequencies from a molecular dynamics simulated time signal [8]. The 
difficulty in the direct inversion by Eq. (3) is that in order to resolve close-lying 
frequencies, one needs to know the autocorrelation fimction (or time signal in general) 
on a very dense time grid over a long period of time [2,3], When the' time signal is 
available only on a set of discrete times and for a short time period, this approach 
becomes inefficient and inaccurate. Several elegant methods, such as the 
filter-diagonalization technique, developed by Neuhauser [9] and modified by 
Mandelshtam and Taylor [10], have been proposed for such problems. 

In the present approach, we begin by inserting a factor, exp[\a (a> -a>o)^], into Eq. 
(2), thereby defining an auxiliary function 



(Note that Eq. (4) can be viewed as a type of Fresnel transform [1 1] an integral kernel 
i:(o),cOo) = exp[iaiu>-(x>^^]). From that viewpoint, the kernel acts on e ^"X^) farther than 
onJ{u>y) Due to the highly oscillatory nature of the factor exp[z^i(a)-o)o>^ for large a 
values, only frequencies sufficiently close to co = cOq contribute significantly to Eq. (4). 
In fact, it can be shown that in the limit a - ~ ijtt;'a,o)o) becomes proportional ioJ{u> ). 
The factor, e^[z^i(o)-o)o)^], can be viewed as a window fimction that acts as a "filter" so 
that the auxiliary fimction, (X^\CL,{i>^, contains information principally at frequencies near 
a> = (Oq. We note that the factor exp[/^(i>-cOo)^] also can be expressed as a matrix element 
of the "free propagator", ex/7[(-i/4a)(dVdo)^)], according to 



and the integration over co in Eq. (4) is equivalent to evaluating the action of 




(4) 



= (7ri/a)i/^(a;|e^^K> = (7rt/a)V2e-45(a;-o;o). 



(5) 
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ex;?[(-i/4a)(dVda)^)] in the continuous "(^''-representation. That is, the introduction of the 
phase exp[ia((Ai-(x)Qf] is equivalent to freely propagating (in the sense of quantum 
propagation) the function e''"y{G) ) for a "duration", l/(4a ), in a generalized tune. This 
enables us to write Eq. (4) as 

C(t;a.a,o) = (-/°)'/^-P(-^^l(e--'/K)). (g) 

The functiony(Wo) then is obtained simply by inverting Eq. (6), 

/(a;o) = (a/^)'/V*^°'e^4c(«;a,a;o). ^'^^ 

This equation indicates that the value of the functiony(cOo) can be computed by freely 
propagating the auxiliary function 4t;a) for a "duration" l/(4a ). However, for this 
equation to be of any use, we need an independent method for calculating the auxiliary 
function from the experimentally (or theoretically) detemiined C(t). The function 
(^t;a,a)o) can be calculated directly from the experimentally determined signal, C(t), 
using the following expression: 



^(*o)]. (8) 



This equation is obtained by deriving (and integrating) the auxiliary partial differential 
equation, 



Thus, our procedure consists of propagating the experimental time signal, C(0 
(multiplied by exp(io)ot)), from t to to over the"duration" a to obtain ^jtto;a,<Oo), and then 
propagating the resulting (:tto;a.<^o) fro™ to o) over the "duration" l/(4a ). Clearly, 
the product of the two "propagation durations" is a constant (axlMa = IM), indicating 
that there exists a reciprocal relationship characteristic of a "time-eneigy"-like uncertainty 
principle. In the numerical example we give below, we shall choose a = l/4a = 1/2; 
however, this is not necessary and the question of the optimum "duration" for each 
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propagation remains to be explored. We also find it numerically efficient to employ the 
DAF-free propagator [4], 




where 



^« = (-l)V[(2^)^/V(0)n!22"]. 



(11) 



and 



ar\a) = 0-2(0) 4- 2,a. 



(12) 



where, <^0) and M are DAF-parameters, and the Hjn's are the Hermite polynomials 
[4,12]. This makes it possible to carry out the two propagations with a single 
matrix- vector multiplication for each propagation. \ It should be noted that this D AF-free 
propagator exactly propagates the DAF representation of a function. 

We believe that the dual-propagation inversion scheme we have outlined has 
several potential advantages. First, the D AF-fitting method on which the scheme is based 
allows us to effectively filter noise while preserving signal. The method provides a 
"signature" as a function of the DAF parameters which gives a measure of signal to noise 
and allows the parameters to be varied to achieve optimum filtering. Second, as abeady 
noted, the propagation of the DAF representation of a function using the Hermite DAF 
propagator is exact. Third, each propagation has associated with it a characteristic width 
controlled by the Gaussian (see Eqs. (10) and (12)) and determmed by the optimum fit 
of the input data. These widths control on what domain oft the function C(t) must be 
known in order to determine a particular value of/coo). Finally, DAFs can be used to 
"pad" (i.e., extend) the initial C(/) dataset in various ways to take advantage of any a 
priori knowledge one might have. 

A detailed comparison of our method with other inversion methods is beyond the 
scope of this brief report and will be the topic of a subsequent communication. \ 
However, we will illustrate the method on a challenging problem, namely extracting a 
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half-sine wave from its discretized and noisy transform. The underlying function, f(a) ), 
in our example is then 



f{u>) = sin{ul (0<a' <7r], ^^^^ 
= 0 otherwise . 



Since f(a) ) has compact support, the time-signal, C(r)(analytically obtained by Eq. (2)) 
of course, cannot have compact support. In fact, it has a very slow decay with increasing 
|^|. First,y(a)) was computed by our dual propagation using Eqs. (7) and (8), and the result 
compared with the truncated sine function, Eq. (13). In Fig. 1, we present the auxiliary 
function, (Xt;a,uiQ), at r==0 and a=l/2, as a function of co^-We have calculated this 
function both by direct numerical integration of Eq.(4) and by DAP propagation using 
Eq. (8) [13]. The resuhs of the DAF propagation and the direct numerical integration are 
essentially identical up to the fourth decimal point. The propagation to generate the 
auxiliary fimction has been carried out by a highly efficient" one-step DAF-propagation" 
of duration a = Vi, Figure 2compares the spectral function,y((*) ), obtained by the present 
dual propagation [13] with the original truncated sine-function. It Asian that the two are 
visually indistinguishable. In this calculation, we assumed discrete "experimental" values 
of C(r) are known from -45^/^45 on a discrete grid with uniform spacing Aro=l/l 8, and 
the propagation from (Oq to co employs the computed values of djtto;ot>^o) fro"^ 
-30^a>o^35,with a uniform grid spacing of Aco^ = 1/18. We emphasize that one only 
needs the time signal C(t) for a finite time interval, [t^ny^^]y in order to calculate 
d3tto;a,coo) using Eq. (8). Although the accuracy increases for longer-time signals, C(r), 
the length of the time interval is controlled the width of the DAF free propagator matrix 
(determined by o/A^q). Thus, short-duration time signals can yield an accurate spectrum 
y(w ) for the frequencies filtered out in Eqs. (4)and (8) in our method. It should also be 
noted that the present technique does not require a knowledge of the system Hamiltoni 
an, or any diagonalization processes, since it makes direct use of the time signal (or 
autocorrelation function). Thus, the measured or computed time signals directly yield 
information in the frequency domain (e.g., optical spectra or normal mode frequencies). 

One of the features of DAFs that has been discussed previously is their ability to 
filter noise from a given function. The free-propagating DAF described here also has this 
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filtering property, which is extremely useful in the present context. When an inversion 
is carried out for experimental data, the presence of noise in the time signal C(t) will 
create inacciu:acies in the calculated spectrum,y(o) ). (This problem is especially serious 
in the inverse Laplace transformation, since statistical errors present in the signal can be 
easily amplified.) In order to illustrate the filtering feature of our DPI procedure, we 
added random noise(up to ± 20\%) to the time signal C(/), and carried out the dual 
propagation inversion as before. We find that the resulting auxiliary function, d[t^;a,uyQ) 
is very similar to that obtained earlier using the noise-firee time signal, confirming that 
high frequency, random noise is automatically removed by the DPI method. Figure 
Sillustrates this denoising feature of the DPI, using Hermite-DAF fi-ee propagators, by 
comparing the spectrum obtained firom the "corrupted" time signal (with ± 20\% noise) 
to that obtained from the noise-fi-ee time signal. Both time domain signals extend over 
the range -45£r^45,with the same sampling fi-equency as before. It is seen that the DPI 
abnost completely removes the effects of the noise in the time signal. We also employed 
a stationary DAF filtering method [14] to remove the noise fi-om the time signal before 
carrying out the DPI procedure, and obtained a firequency domain spectrumy(o> ) that is 
visually indistinguishable firom that depicted in Fig. (3). 

Finally, we explore the consequences of a more severe truncation of the 
time-domain signal, and therefore carried out the DPI procedure using C(t) over the 
artificially shortened interval -5 ^t^5. In these calculations, we have treated the noiseless 
C(t), Our results are shown in Fig. 4, where we compare the inversion result to the exact, 
original signal. We see that there is relatively little error induced (comparable to that due 
to introducing noise; see Fig. 3). A common procedure to decrease the aliasing effects 
caused by the truncation of the time domain signal is to artificially damp C(t) to zero 
beyond certain times. This is typically done by interpolation or by smoothly joining an 
exponential decaying analytical "tail". In the case of DAFs, we can add a gap on either 
side of the truncated C(0, and use DAF-fitting to "fill in" the missing gap values, thereby 
joining the actual data to an analytical tail function. In the present study, we introduced 
the gaps 5^|t|^7.5, and joined the known data between -5*:st^5 to the analytical tail 



C{t) = 



-0.040648 exj7(-0.107871 



(14) 
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The results of the DPI are compared to the original signal in Fig. 5, and we see that while 
some aliasing still exists, it is reduced over the entire range of (O . Similar results are 
obtained when noisy signals are truncated, and then DAF-joined to an analytical decaying 
tail function. We conclude by noting that we expect even better results will be obtained 
by using DAFs to periodically (or in other ways) extend the truncated time domain 
signal. This will be reported elsewhere. 

We conclude by briefly considering the Laplace transform problem, namely 
numerically inverting 

e-/H . (15) 

which our method (in principle) also solves. Here C(/) is analytic in the half-plane (t>0). 
In the standard way, it is convenient to introduce a new function 



/(w) = f(uj) for a; > 0 
— 0 for w < 0 

to write 



J — no 



(16) 



Exactly paralleling our previous discussion we then obtain 

7K) = (a/«)'/Vo'c*4c(<;a,u,o). (17) 

where now 

C(t;a,wo)= / e'-t"- -)'e-"7(y) (18) 

t/ —CO 

The quantity djtt;a,(Oo) obeys the equation 
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subject to the initial-value boundary condition ^t;a=0,o)o)=C(0. Equation (19) is 
identical to Eq.(9) except for the sign of the right-hand side. The inversion then proceeds 
by solving Eq.(19), subject to the boundary condition at a =0 and propagating the result 
via Eq.(l 7) to obtain the answer. Although this solves the problem in principle, the first 
step(that of solving Eq.(19) to acceptable accuracy when C{t) is imprecisely known) is 
a challenge. In the Fourier transform case, we solved the corresponding partial 
differential equation by means of a second propagation, and indeed we can still write the 
formal solution to Eq.(19) as 

(20) 

since C(0 is analytic in the half-plane (/>0) and hence its derivatives to all orders 
uniquely exist in the half-plane. However, using the D AF propagation scheme (at least 
in a straight-forward manner) presents difficulties near the origin because of the 
singularity in C(t) at t=0. Various stratagems can be devised for attacking these problems, 
and they are currently under investigation. 
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DISTRIBUTED APPROXIMATING FUNCTIONAL WAVELET NETS 
INTRODUCTION 

For real-world signal processing, pattern recognition and system identification, 
information extraction from a noisy background is the fundamental objective. To obtain 
an ideal output vector Y(X, W) from the observation input vector the system (neural 
network) should possess the following two kinds of smoothness [6] (where W is the 
response or the entry of the system, normally for neural networks, called the weight 
vector). 

(a) Functional space smoothness 

(b) State space smoothness 

The degree of smoothness in functional space governs quality of the filtering of the 
observed (noisy) signal. The smoother the output signal, the more noise is suppressed. 
State space smoothness implies that a weak fluctuation of the weight vector W={>v(/), 
i=0, i-1 }, has a small effect on the output signal, which makes the system less sensitive 
to the input distortion. 

For a robust estimation system, the output not only approaches the observed 
signal value, but also smoothes the signal to suppress the distortion due to noise. 
Simultaneously, the state space should be smooth to ensure stability. Based on these 
facts, one finds that the least mean square (LMS) error 



the regularization constraints of order r 



£. = J[%^?^. (2) 

and the condition of the system 



I 

are the three dominant factors that must be accounted for in designing a robust estimation 
system. 
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Distributed Approximating Functionals (DAFs), which can be constructed as a 
window modulated interpolating shell, were introduced previously as a powerful grid 
method for numerically solving partial differential equations with extremely high 
accuracy and computational efficiency [3,9,10]. In this paper, we use the DAF 
approximation scheme for implementing a neural network. 

Compared with other popular networks, DAF nets possess advantages in several 

areas: 

(1) a DAF wavelet is infinitely smooth in both time and frequency domains. 

(2) For essentially arbitrary order of the Hermite polynomial, the DAF shells 
possess an approximately constant shape, while commonly used wavelet functions 
always become more oscillatory as the regularization order is increased.; 

(3) The translation invariance of the DAF approximation ensures feature 
preservation in state space. The signal processing analysis can be implemented in a space 
spanned by the DAFs directly. 

(4) Complicated mathematical operations, such as differentiation or integration, 
can be carried out conveniently using the DAF interpolating shell. 

(5) The identical smoothness of the DAF wavelet space and DAF state space 
underlie the inherent robustness of the DAF wavelet nets. 

REGULARIZED DAF WAVELET NETS 

In general, signal fihering may be regarded as a special approximationproblem 
with noise suppression. According to DAF polynomial theory, a signal i^proximation 
in DAF space can be expressed as 

where the Sj[x) is a generalized symmetric Delta functional. We choose it as a Gauss 
modulated interpolating shell, or the so-call distributed approximating functional (DAF) 
wavelet. The Hermite-type DAF wavelet is given in the following equation [10]. 
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The function H^^ is the Hermite polynomial of even order In. The qualitative behavior 
of one particular Hermite DAF is shown in Fig.l. 

The DAF wavelet neural nets possess the alternative feature of the commonly 
used DAF approximation as 

The weights of the nets determine the superposition approximation £{x) to the 
original signal g{x) eL\R). It is easy to show that the weights of the approximation nets, 
w(i), are closely related to the DAF sampling coefficients g(jc-). The irregular finite 
discrete time samplers of the original signal are selected for network learning. If the 
observed signal is limited to an interval / containing a total ofN discrete samples, /={0, 
1, . . ., A^-1 }, the square error of the signal is digitized according to 

L'(«('»)-i(")l^ (7) 

This cost function is commonly used for neural network training in a noise-free 
background and is referred to as the minimum mean square error (MMSE) rule. 

However, if the observed signal is corrupted by the noise, the network produced 
by MMSE training causes an unstable reconstmction, because the MMSE recovers the 
noise components as well as the signal In this case, the signal-noise-ratio (SNR) cannot 
be improved much. Even for a noise-free signal, MMSE may lead to Gibbs-like 
undulations in the signal, which is haimfiil for calculating accurate derivatives. Thus, for 
more robust filtering, the network structure should be modified to deal with the particular 
situation. In this paper, we present a novel regularization design of the cost function for 
network training. It generates edge-preserved filters and reduces distortion. To define the 
regularity (smoothness) of a signal, we introduce a "Lipschitz index" [6]. 

Definition 1 : Let J{x)eL\R\ for any a>0, then if 

the signaiy(jc) is said to be unified Lipschitz in the space L\Ry The constant, is the 
Lipschitz index o{J(x). 
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It is easy to show that when the Lipschitz index, a, is an integer, the Lipschitz 
regularity is equivalent to the differentiability of/x) with same order. For commonly 
used signals, the Lipschitz index ^0. In the presence of noise distortion,^ the Lipschitz 
index always satisfies ia<-l. This is because the noise causes sudden phase changes at 
neighboring points. To eliminate non-ideal imdulations, we need to preserve the signal 
trends while making a small MSE approximation. To achieve this, an additional smooth 
derivative term, is introduced to modify the original cost function. The new cost 
function is then 

^=£^+^^ 

= 2:feW-iW]2..;iJ^[igfij2^ (9) 

The factor / introduces a compromise between the orders of approximation and 
smoothness. Generally, the derivative order r>2 is used to evaluate the smoothness of the 
signals. Using the properties of the Hermite D AFs, we find that the derivative term of the 
regularized cost function can be expressed in a comparatively simple convolution 
form. 



where A is termed a "differentiating DAF" and is given by 




It is exactly the rth derivative of 6o, Because of the infinite smoothness of DAF 
wavelets, any order derivative can be obtained. The Hermite polynomial H\% generated 
by the usual recursion 



I. rt = 0 

2xff„»i(x)-2(/i-l)//„_2(Jc). ">> 



(12) 
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The predominant advantage of the Hermite polynomial approximation is its high-order 
derivative preservation (which leads to a smooth approximation). 

To increase the stability of the approximation system further, -an additional 
constraint in state space is taken to be 

Thus the complete cost function utilized for DAF wavelet net training is given by 



SIMULATIONS 

Two biomedical signal processing applications (for electrocardiogram and 
electromyography) using the DAF wavelet neural nets are presented in this chapter. 

Automatic diagnosis of electrocardiogram (ECG or EKG) signals is an important 
biomedical analysis tool. The diagnosis is based on the detection of abnormalities in an 
ECG signal. ECG signal processing is a crucial step for obtaining a noise-free signal and 
for improving diagnostic accuracy. A typical raw ECG signal is given in Fig. 2. The 
letters P, Q, R, S, T and U label the medically interesting features. For example, in the 
normal sinus rhythm of a 12-lead ECG, a QRS peak follows each P wave. Normal P 
waves rate 60-100 bpm with <10% variations. Their heights are <2.5iiHn and widths 
<0.1 Is in lead 11. A normal PR interval ranges from 0.12 to 0.20s (3-5 small squares). A 
normal QRS complex has a duration of <0. 1 2s (3 small squares). A corrected QT interval 
(QTc) is obtained by dividing the QT interval with the square root of the preceding R-R' 
interval (normally QTc=0,42s). A normal ST segment indicates no elevation or 
depression. Hyperkalaemia, hyperacute myocardial infarction and left bundle can cause 
an extra tall T wave. Small, flattened or inverted T waves are usually caused by 
ischaemia, age, race, hyperventilation, anxiety, drinking iced water, LVH, drugs (e.g. 
digoxin), pericarditis, PE, intraventricular conduction delay (e.g. RBBB) and electrolyte 
disturbance [12]. 

An important task of ECG signal filtering is to preserve the true magnitudes of 
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the P, Q, R, S, T, and U waves, protect the true intervals (starting and ending points) and 
segments, and suppress distortions induced by noise. The most common noise in an ECG 
signal is AC interference (about 50Hz-60H2 in the frequency regime). Traditional 
filtering methods (low-pass, and band-elimination filters, etc.) encounter difficulties in 
dealing with the AC noise because the signal and the noise overlap the same band. As a 
consequence, experienced doctors are required to carry out time-consuming manual 
diagnoses. 

Based on a time varying processing principle, a non-linear filter [4] was recently 
adopted for ECG signal de-noising. Similar to the selective averaging schemes used in 
image processing, the ECG is divided into different time segments. A sample point 
classified as "signal" is smoothed by xising short window averaging, while- a "noise" 
point is treated by using long window averaging. Window width is chosen according to 
the statistical mean and variance of each segment. However, this calculation is 
complicated and it is not easy to select windows with appropriate lengths. The 
regularized spline network and wavelet packets were later used for adaptive ECG 
filtering [5, 6], which is not yet efficient and robust for signal processing. In our present 
treatment, regularized DAF networks are used to handle a real- valued ECG signal. We 
utilize our combined group-constraint technique to enhance signal components and 
suppress noise in successive time-varying tilings. 

The raw ECG of a patient is given in Fig. 3(a). Note that it has typical thorn-like 
electromagnetic interference. Fig. 3(b) is the result of a low-pass filter smoothing. The 
magnitudes of the P and R waves are significantly reduced and the Q and S-wraves ahnost 
disappear completely. The T wave is enlarged, which leads to an increase in the QT 
interval. Notably, the ST segment is depressed. Such a low-pass filtering result can cause 
significant diagnostic errors. Fig. 3(c) is obtained by using our DAF wavelet neural nets. 
Obviously, our method provides better feature-preserving filtering for ECG signal 
processing. 

Another application is for electromyography (EMG) filtering. Surface EMG has 
been used to evaluate muscle activation patterns in patients with gait disorders since the 
mid 1900s. In experimental as well as routine recording of muscle action potentials, 
signal cross-talk from various sources cannot always be avoided [2], In particular EMG- 
investigations within the areas of physical science, orthopedics or ergonomics, where the 
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collection of data has to be carried out under field conditions, the measured signals are 
often incorrect due to movement of the subject. In particular DC ofF-set-voltages, 
movement of electrodes and cables, 50 Hz interference and electrostatic interference 
should all be considered. But even with the utmost case, movement artifacts, particularly 
in studies of movement, cannot be completely avoided. Thus for a number of quantitative 
signal processing procedures, an elimination of interference has to be carried out. 

Once the raw scanning EMG data are stored in the computer, several processing 
options are available for improving the signal quality. The first one is removing the DC 
offset per recorded trace for elimination of movement artifacts. The second option is a 
moving window smoothing (three points or more) in the time-direction of each trace for 
noise reduction. The third, a nonlinear median filtering over 3, 5 or 7 points is-employed 
in the depth direction for elimination of non-time-locked activity. The iteration times of 
median filtering are performed, depending on the quality of the recorded scan [2], 
Although a median filter is better for impulse-like noise removal than linear filters (low- 
pass, high-pass or band-limited) [1 1], it is not optimal since it is typically implemented 
uniformly across the image. The median filter suppresses both the noise and the true 
signal in many applications. It results in amplitude reduction of the sharp signal peak, 
which is deleterious for diagnostic analysis. 

In this paper, we use a DAF wavelet neural network for the adaptive EMG 
processing. The sampling is irregular to match the time-varying characteristics of EMG. 
Additional regularization and robust designs enable the optimal smooth approximation 
of the signal. As shown in Fig.4, the original measured EMG has many thom-like noise 
peaks (Fig.4(a)). A simple low-pass filtered result is shown in Fig.3(b). Our newly 
developed technique results in the solution shown in Fig.4(c), Again, one should note the 
presentation of feature details achieved. 

CONCLUSION 

Regularized DAF wavelet neural networks are proposed for non-stationary 
biomedical signal processing. The DAF approximation shells possess infinite smoothness 
in both physical and firequency domains, which enable the high-resolution time-varying 
analysis of the signal. The optimal signal filtering solution is obtained using a 
combination of several different contributions to a '*cost function". Measured ECG and 
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EMG signals are employed for testing the new technique. The simulations show that our 
method is both efficient and robust for time-varying filtering. 
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PERCEPTUAL NORMALIZED SUBBAND IMAGE RESTORATION 



INTRODUCTION 

An earlier group normalization (GN) technique [1] is extended to account for human 
perceptual responses. The purpose of the GN is to re-scale the magnitudes of various 
subband filters and obtain normalized equivalent decomposition filters (EDFs). Our 
human visual system can be regarded as a natural signal and image processing devise— a 
non-ideal band pass filter. In order to achieve the best noise-removing efficiency, human 
vision response [2] is accounted for by a perceptual normalization- (PN). These 
approaches are combined with our biorthogonal interpolating filters [3]. tp achieve 
excellent perceptual image restoration and de-noising. 

The biorthogonal interpolatmg filters are constructed by Swelden's lifting scheme 
[4] using Gaussian-Lagrange distributed approximating functionals (DAFs) [5] which 
were introduced as an efficient and powerfiil grid method for numerically solving partial 
differential equations with extremely high accuracy and computational efficiency. From 
the point of view of wavelet analysis, GLDAFs can be regarded as smooth low pass 
filters. 

PERCEPTUAL NORMALIZATION 

The main objective of wavelet analyses is to provide an efficient L\R) 
representation of a fimction. For signal and image processing, it is important to preserve 
the signal components of the corresponding subband filters. Wavelet coefficients can be 
regarded as the output of the signal passing through the equivalent decomposition filters 
(EDFs). 

The responses of the EDFs are the combination of several recurrent subband 
filters at different stages. Wavelet coefficients can be regarded as the output of the signal 
passing through the equivalent decomposition filters (EDF). The responses of the EDF 
are the combination of several recurrent subband filters at dififererit stages.. As shown in 
Fig. 1, the EDF amplitudes of our DAF-wavelets in each sub-blocks are different. Thus 
the magnitude of the decomposition coefficients in each of the sub-blocks carmot exactly 
reproduce the actual strength of the signal components. To adjust the magnitude of thei 
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response in each block, the decomposition coefficients are re-scaled with respect a 
common magnitude standard. Thus EDF coefficients, CJJc), in block m should be 
multiplied with a magnitude scaling factor, yl„, to obtain an adjusted magnitude 
representation [1]. This idea was recently extended to Group Normalization (GN) of 
wavelet packets for signal processing [1]. 

The main objective of wavelet signal filtering is to preserve important signal 
components, and efficiently reduce noisy components. To achieve this goal, we utilize 
the human vision response to different frequency bands. An image can be regarded as the 
impulse response of the human visual system. The latter essentially consists of a series 
of "subband filters". It is interesting to note that, just like the non-uniform response of 
wavelet subband EDFs, the human "subband filters" also have non-uniform-responses 
to a real object. Using a just-noticeable perceptual matrix, we can efficiently remove the 
visual redundancy fi-om decomposition coefficients and normalize them with respect to 
the standard of perception importance. A mathematical model for perception efficiency 
has presented by A. B. Watson, etc. [2] and can be used to construct the "perceptual 
lossless" quantization matrix for normalizing visual response [2]. This treatment 
provides a simple human-vision-based threshold technique [6] for the restoration of the 
most important perceptual information in an image. For gray-scale image processing, the 
luminance (magnitude) of the image pixels is what we are most concerned with. The 
combination of the above mentioned group and visual response normalization can be 
called the Visual Group Normalization (VGN) of wavelet coefficients. 

BIORTHOGONAL INTERPOLATING DAF WAVELETS 

One of most important aspects of wavelet transforms is the wavelets themselves. 
From the point of view of applications, the stability, regularity, time-frequency 
localization and computational efficiency are the most important criteria for selecting 
wavelets. Interpolating wavelets are particularly efficient for basis set construction since 
their multiresolution spaces are identical to the discrete sampHng spaces. In other words, 
there is no need for one to compute wavelet expansion coefficients by the usual inner 
products. This makes it easy to generate subband filters in a biorthogonal setting without 
requiring tedious iterations. Moreover, adaptive boundary treatments and non-uniform 
samplings can be easily implemented using interpolating methods. The lifting scheme 
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discussed by Swelden [4] is used in this work for constructing interpolating filters. 
We use the interpolating Gaussian-Lagrange DAF (GLDAF) [5] 

= KM fl — 

as a scale function for our wavelet construction. Here W^x) is a window function and is 
selected as a Gaussian because it satisfies the minimum frame bound condition in 
quantum physics. The quantity cris a window width parameter, and Pj^x) is the usual 
Lagrange interpolation kernel Biorthogonal dual DAF scaling functions and DAF- 
wavelets are constructed using Swelden's Ufling scheme [4]. The present DAF-wavelets 
are extremely smooth and rapidly decay in both time and frequency domains. As a 
consequence, they are free of Gibbs oscillations, which plague most conventional wavelet 
bases. As plotted in Fig. 2, our scaling functions and wavelets display excellent 
smoothness and rapid decay. 

DEMONSTRATIONS 

We use as benchmarks the 512x512 Y-component Lena and Barbara images to 
demonstrate the restoration efficiency of the present approach. The so-called "Lena" 
image possesses clear sharp edges, strong contrast and brightness. Thelhigh texture 
component and consequently high frequency edges in the Barbara -image create 
considerable difficulties for many commonly used filtering techniques. A simple low- 
pass filter will not only smooth out the high frequency noise but also blur the image 
edges, while a simple high-pass filter can preserve the texture edges but will also cause 
additional distortion. 

As shown in Fig. 3(a) and Fig. 4(a), the original Lena and Barbara images are 
each degraded by adding Gaussian random noise. The median filtering (with a 3x3 
window) result is shown in Fig. 3(b) and Fig. 4(b), which is edge-blurred with low visual 
quaHty. It is evident that our perceptual DAF wavelet technique (Fig. 3(c) and Fig. 4(c)) 
yields better edge-preservation and provides high visual restoration performance. 



wo 99/46731 



PCTAJS99/05426 



187 

REFERENCES 

1 Zhuoer Shi, Z. Bao, "Group-nonnalized wavelet packet signal processing", 
Wavelet Application IV, SPIE, Vol. 3078, pp.226~239, 1 997 

2 A. B. Watson, G. Y. Yang, J. A. Solomon, and J. Villasenor "Visibility of wavelet 
quantization noise," IEEE Trans. Image Proc, vol. 6, pp. 1 164-1 175, 1997. 

3 Zhuoer Shi, G. W. Wei, D. J. Kouri, and D. K. Hoffinan, "Perceptual Image 
Processing Using Gaussian-Lagrange Distributed Approximating Functional Wavelets," 
submitted to IEEE SP Letters, 1998. 

4 W. Swelden, "The lifting scheme: a custom-design construction of biorthogonal 
wavelets," Appl. And Comput. Harmonic Anal., vol.3, no.2, pp.l86~200, 1996 

5 G. W. Wei, D. S. Zhang, D. J. Kouri, and D. K. Hof&nan, "Lagrange distributed 
approximating Functionals," Physical Review Letters, Vol. 79, No.5, pp. 775-779, 1 997 

6 D. L. Donoho, "De-noising by soft-threshold," IEEE Trans. Information Theory, 
vol. 41, no.3, pp. 613-627, 1995 



wo 99/46731 



PCT/US99/05426 



188 

QUINCUNX INTERPOLATING 2D AND 3D WAVELETS-DAF 
INTRODUCTION 

Nowadays, wavelets has became an important area of research for image 
processing; however, most of the work done until today, it has been concentrated for the 
one dimensional case while the multi-dimensional case has been approached using the 
tensor product or known as well as separable wavelet transform. In this work we discuss 
a way for constructing "non-separable" wavelets in two and three dimensions using 
distributed approximating fimctionals (known as DAF) . A recent work [dafD04] 
pubUshed in 1998 proposes a way of constructing wavelets in one dimension called 
wavelets-DAFs. This work proves that this kind of wavelets improves the accuracy for 
solving linear and non-linear partial differential equations. Using a; non-separable 
sampling known as Quincunx for the 2D case and FCO for the 3D case (or Quincunx in 
3D) allows us to have a better perception for the human visual system. The first section 
of this paper will explain the Quincunx sampling in 2D and 3D (FCO). Then, the second 
section addresses the construction of the scaling function. The third section explains the 
way of getting the actual values for the interpolating multidimensional filter. 

NONSEPARABLE LATTICES 

The quincunx lattice is considered as the simplest way of sampling in two 
dimensions. We need in the 2D case two vectors to define the sampling, one possible 
value for those vectors is 



d = 



and 



d = 
2 



I- 1 



which lead to the following dilation matrix 
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(1) 



In the Quincunx lattice, the subsampling process rotates the image by 45° and flips the 
image about the horizontal axis. 

As you can see in figure 1 , the quincunx lattice has a checkerboard pattern. FCO 
lattice is used for the 3D case. The FCO lattice belongs also to the general checkerboard 
lattices. One of the possible dilation matrices for the FCO is 



0 0 n 

1 1 0 

lo 1 V 



(2) 



notice that det(D)=M=2, for either quincunx or FCO, 
Scaling function 

Due to the fact that the scaling function behaves as generalized delta sequences 
we use the DAF approach to generate it. The two most important aspects in the design 
of the scalmg function are the decay rate and smoothness. We studied the following 
functions and their properties. First, the normal function in 2D defmed a&Tollows 



N(x,y) = e 




(3) 



which obviously has a Gaussian decay, and its sigma defines the width of the 
neighborhood points. Another, function considered because of its properties under the 
Fourier transform is the sine equation defined as follows 
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sincdx +y 




(4) 



The combination of these functions produce the father wavelets which are both smooth 
and rapidly decaying 



The next section explain the domain of x and jy for equation 5. Figure 2 shows the scaling 
function for the quincunx lattice when a=a=l. Equation 5, it is clearly a nonseparable 
function. 

CONSTRUCTION OF THE QUINCUNX INTERPOLATING FILTER 

We want to interpolate the red dot shown on figure 3, which coordinates are 
(1/2,1/2). The plane is the one that we obtain after we downsample the original lattice 
using quincunx. Then, this xy plane was rotated by 45° with respect to the original lattice, 
and it was flipped about the horizontal axis. 

The circles in figure 3 represent, the values known from function. Those values 
will be used to approximate the value of the function at the red point (l/2;i/2) called P. 
Those points are labeled depending on the distance with respect to P. For instance, all 
points labeled "1" have the same distance to P, (2)"^*^\ Those points represent the filter 
taps. There has been some work on calculating the filter taps for the quincunx in 2D and 
3D, already; however, those filter taps are fix for the point at P at (1/2,1/2). In our work, 
we recalculate the filters depending on where in the grid is your new point P. The know 
values shown on figure 3 are fix, but not their labels. For instance, if our point P is 
located at (1/3,1/3) the values of the filter taps in the grid will change as shown on figure 
4. 

This is something different with respect other works in this area. We calculate the 
value of those filter taps using equation 5, where x and are the difference between the 




2 



(5) 
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points on the grid and the point P. The summation of all the values of the filter taps must 
be 1 . In the case when we have only four points to approximate P, when P is located at 
(1/2,1/2) the result is trivial since we have four taps labeled one;therefore they must have 
the same value. Since their summation must be equal to one we resolve that those taps 
T(l)=l/4. However, when we have more points to approximate P, this is not that trivial. 
Using the D AF theory we have to look for the optimal sigma and alpha to estimate those 
filter taps. We could even consider different values of sigma depending on which points 
on the grid we are using to predict the value of the fimction at P. 

The selection of the scaling fimction was crucial, thinking about computational 
time. Moreover, since we use DAFs to calculate the filters taps equation 5 allows us to 
interpolate the n-derivate of the fimction at point P. 
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DAF APPROArH TO TMAGE RESTORATION 

INTRODUCTION 

Anisotropic diffusion using nonlinear partial differential equations (PDEs) was 
proposed by Perona and Malik [1] for removal of noise, edge detection, and image 
enhancement. Considerable theoretical and numerical analyses and improvements have 
been carried out since its proposal [1-12], In [1], the restored image is obtained by 
evolving the original noisy image Uq{x^) according to the following diffusion equation 
with a variable diffusion coefficient 



where, the initial condition is M(x,7,0)==Uo(Xjy), where div defines the divergence 
operator, and c(lVu|) is a nonincreasing positive diffusion coefficient function. It is 
interest to see that when c is a constant C, (1) reduces to the heat conduction or isotropic 
diffusion equation, as follows 



This equation has been used to identify objects in image processing [13]. However, 
using the isotropic diffusion for image restoration results in a severe edge=blurring 
problem, which is the principal reason for introducing anisotropic diffusion. The basic 
idea of anisotropic diffusion is: [1] for a region with small gradient |Vu|, the diffusion 
coefficient c will be large to encourage smoothing this region; for a large |Vu| region, c 
will be small to discourage the diflfusion for the purpose of preserving the edges. In [1], 
Perona and Malik suggested the following two expressions for the diffusion coefficient, 



di 



= div[c(|Vu(x,y,t)|)Vtt(x,y,t)]. 



(1) 




(3) 



and 
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c(|Vtt|) = 



1 



(4) 




2» 



1-f 



where k is a constant used to provide a threshold for noise removal, i.e., to enhance the 
edges and smooth the homogeneous areas. However, as pointed out by many authors 
[2,4,6,8], such a procedure (1) with diffusion coefficients given by (3) or (4) is ill posed. 
When the image is corrupted with Gaussian noise, it can occur that the gradient 
produced by noise is comparable to that produced by the edges. Application of the 
anisotropic diffusion equation to such an image will tend to preserve both the image 
edges and the undesirable noisy edges simultaneous. Nevertheless, in order for the 
solution of the diffusion equation to be unique, lVu|c(|Vu|) must be riondecreasing. 
Otherwise, the solution may diverge in time for some choices of numerical schemes. The 
same image with a minor difference amount of noise may lead much different solutions 
using the same numerical scheme. The solutions of different niunerical schemes to the 
same image with the same amount of noise may also have much difference. To alleviate 
the ill-posed problem, it is crucial to detect the edges as accurately as possible, but the 
edges of a image are usually unknown to us. For this purpose, a selective smoothing of 
the diffusion coefficient in (1) was introduced in [2], as follows: 



where g(x^) is a Gaussian fiUer to detect the edges and g(x j/) */(jc denotes the 
convolution operation at a given time t. This smoothing operation is not only limited to 
the use of a the Gaussian filter. In [9], a symmetric exponential filter was proposed, and 
was claimed to be more accurate for detection the position of edges. It is given by 



Using this smoothing filter, significant improvement in the quality of the restored images 
was attained [9]. However, equation (5) has a serious drawback of requiring compution 
of the convolution because it is required at each time step [8]. 

We disclose using a recently developed distributed approximating fiinctional 
(DAF) approach [15-17] to solve the anisotropic diffusion problem. The DAFs used here 



du{x,y,t) 
di 



= div{c[|V(g(x,y)*ti(x,y,i))l] 
X Vu(x.y,t)}, 



(5) 



(6) 
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have several advantages. First, they can be used to spatially discretize the diffusion 
equation. Previous applications of the DAFs to nonlinear partial differential equations 
(PDEs) has shown that they are very accurate and stable PDE solvers [18-19]. Second, 
by proper choice of the parameters, the DAF evaluation of \Vf\ in the diffusion 
coefficient automatically produces smoothing. Third, a periodic boundary condition is 
added to the image, using DAFs according to a recently developed algorithm [20,22], 
making it possible to utilize DAFs and many other numerical schemes which require a 
knowledge of the boundary conditions. Fourth, the DAF can be used as an approximate 
ideal low pass filter to remove the high frequency noise before performing the time 
evolution. The success of DAFs applied to the anisotropic diffusion equation is due to 
their so called "well-tempered" property, which is the most important feature in the DAF 
theory [17]. 

DISTRIBUTED APPROXIMATING FUNCTIONAL FORMALISM 

The distributed approximating functionals (DAFs) were introduced as 
"approximate identity kemels" used to approximate a function and its various derivatives, 
in terms of a discrete sampling [15-17]. One of the most important properties of one class 
of commonly used DAFs is the so-called "well-tempered" approximation, which 
distinguishes them from many other numerical schemes. The "well-tempered" DAFs are 
constructed such that there are no "special points" in the DAF approximation; /.e, the 
approximation to a function has a similar order of accuracy both on and off the input grid 
points. The DAFs also provide a "well-tempered" approximation to the derivatives. As 
long as the function and its derivatives remain in the "DAF-class", the "well-tempered" 
property will ensure that they have similar accuracy. In contrast, many other numerical 
schemes (e.g., basis expansions, wavelets, splines, finite differences, finite elements, 
etc), yield exact results for the function on the grid pomts, but typically at the expense 
of giving significant worse results for the function off the grid points. Such interpolative 
methods are always less accurate for the derivatives of the function. The drawback of the 
interpolative methods is especially distinct when the function is contanmiated by noise, 
since the noise in the signal input data will be kept unchanged. In this case, a filter must 
be used to eliminate the noise in advance. Another feature of certain DAFs is that since 
they are approximate integral identity kemels, they yield an integral representation of 
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differential operators. By proper choice of the parameters, the DAFs provide a 
controUably accurate analytical representation of derivatives on and off the grid points, 
which is crucial to their success in many applications. One of the-^most useful 
realizations of the "well-tempered" DAFs is the Hermite DAP, given by 

where z={x-xy( ^ a), HjnCz) is the usual (even order) Hermite polynomial, and amd 
M are the DAF parameters. The HDAF is dominated by its Gaussian factor exp(-z\ 
which serves to control the effective width of the function. In the limit of either M - «> 
or o - 0, the Hermite DAF becomes identical to the Dirac delta ftmction. A functiony(;c) 
is approximated by the Hermite DAF according to 



/(») ^ foAFix) = f 6m{x - xV)/(x')di 



(8) 



For function sampling on a discrete, uniform grid, the Hermite DAF approximation to 
the fimction at any point x (on or off the grid point) is given by 



fDAF{x) = AJ2^m{x - xj\a)f{xj), 



(9) 



where A is the uniform grid spacing. Only terms in the sum from grid points sufficiently 
close to X contribute significantly. 

Approximations for various linear transformations of a function can also be 
generated using the Hermite DAF. Especially important cases are derivatives to various 
orders, given by 



J-oo 



(10) 



where d^jf^^x-xld) is the 1th derivative of ^(x-x]d), and 



01) 

(11) 
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These expressions can be discretized by quadrature. 



Equation (1 1) and (12) provide convenient methods for calculating derivatives in solving 
nonlinear partial differential equations (NPDEs). 

The HDAF ^proximation can be easily extended to a multi-dimensional case as 
a direct product. For two-dimension &mci\onf[x^\ one has 

ii 

>^ E*>^.(i'-yi,K)/(xi..%j ^^^^ 

and 



J I 

where fDAF^'^^(-^*y) denotes 1th partial derivative and mth partial derivative of the function 
with respect to x and y respectively. 

The Hermite DAF has the so-called "well-tempered" property [17], and it is a low 
pass filter. In Fig. 1, we show plots of Hermite DAFs, obtained using two different set 
of parameters, in (a) coordinate space and (b) fi-equency space, respectively. Their first 
order derivatives are plotted in Fig.l (c), in coordinate space. The solid line is for 
<7=3.54, Af=120 and the dashed line is for o=2.36, M=30. The solid line is said to be in 
the less "well-tempered" or more "interpolative" regime, and the dashed- line is in the 
more "well-tempered" (smoothing) regime . In contrast to the sine function sin(wjc)/7tx, 
with w=7c/A (the ideal interpolator) being the ideal interpolation and vi<7c/A being the 
low pass filter, the Hermite DAF is highly banded in coordinate space due to the 
presence of the Gaussian factor, which means that only a small number of values are 
required on each side of the point x in (1), as is also true for the derivatives of the 
Hermite DAF. This is clearly seen in Figs. 1(a), and 1(c). With proper choice of 
parameters, the Hermite DAF is an excellent filter (see the dashed.line in Fig. 1). For the 
case far away firom interpolation, the DAF approximation is not exact on grids (The 
Hermite DAF is not equal to I at the origin and not equal to 0 at nA, n^O). The DAF 
approximation to the function depends not only on the function value at the grid point 
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itself, but also on values at the grid points close to it. The approximation to a function 
has about the same order of accuracy both on and off the grid points. This 
"well-tempered" property of DAFs plays an important role in our periodic extension 
algorithm. Application of the "well-tempered" DAF to approximate the function and its 
derivatives is relatively insensitive to the presence of noise. As will be seen in this p^er, 
the "well-tempered" Hermite DAF serves as a very good smoothing filter to accurately 
detect the image edges. Application of the "well-tempered" Hermite DAF to the variable 
of the diffusion coefficient in the anisotropic diffusion equation (1) avoids the need for 
an additional smoothing operation. The Hermite DAF will perform the discretization and 
the smoothing operation on the function simultaneously. Since the "well-tempered" 
Hermite DAF is also a low pass filter, by proper choice of the DAF parameters, (13) can 
be used to eliminate the high frequency noise of a signal once the boundary condition 
is known. However, the boundary condition is usually unknown. In the following 
section, we will give a brief review of how a periodic boundary condition is imposed, 
using the DAFs 

DAF-BASED METHOD OF PERIODIC EXTENSION 

For a signal of finite length, it is common requirement for many numerical 
schemes that one know the boxmdaiy conditions of the signal in order to avoid "aliasing". 
Take the noncausal finite impulse response filters (FIRs) as an example. In order to get 
the desired filtering results, the noncausal FIRs require a knowledge of the'signal values 
in the future, which are impossible to obtain in the advance. Also, there are many 
numerical analysis algorithms for which it is computationally efficient for the signal to 
be a given length, which is not usually satisfied in experimental results. In this section, 
we show how our algorithm is employed to impose a periodic boundary condition for the 
signal. The algorithm of periodic extension has been discussed in detail in [20[ and [22]. 
We will give a brief description on how it is implemented. 

The method is closely related to the DAF extrapolation algorithm presented in 
[21]. Assume a limited length of a signal \{f„f2,...,fj.i} is known, and we require it to be 
periodic with the period K so that K-J+1 values \{fj,fj+„...,fK\} are left to be determined. 
Since now the signal isperiodic, the values \{fK+„fK+2v.-,fK+M} are also known and equal 
to \{fi,f2,...,fj.i} respectively. Under the assumption that the signal is in the "DAF-class"-, 
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the unknown values from fj to can be determined by minimizing the following "cost 
function", constructed using the "well-tempered" Hermite DAF: 

where Wp is a weight related to the grid point Xp and f_{DAF}(Xp) is the DAF 
approximation to the function at Xp. The summation range for "p" is selected according 
to the problem considered. From (9) and (15), we have 



/(Xp)- E «Af(x.-xa^)/(xoj 
(=p-« 



(16) 



where w is the DAF half bandwidth. We minimize the "cost function" with respect to the 
variation of the unknowns function according to 

-i^ = 0, J<l<K. 

df{xi) ' - - (17) 

Equations (16) and (17) generate a set of linear simultaneous algebraic equations, given 
by 

X) 2 /(xp) - 6m{xp - xtk)/(xt) 1 

X (V-^Af(xp-x,l«r)) = 0, J<l<K (18) 

where Jp, is the kronecker delta function. These linear algebraic equations-can be solved 
by standard algorithms [18] to predict values of the function in the gap. 

Note that it is the "well-tempered" property of the DAFs that makes the above 
algorithm possible. For interpolation algorithms, the value on each grid point is exact and 
does not dependent on the values of the function at any other grid points, which means 
that the "cost function" is always zero for points on the grid. In contrast, the 
"well-tempered" DAFs are not required to give a exact representation on the grid points. 
The approximate values on a grid point are related to the nearby values. - 

We must stress that this periodic extension of an nonperiodic signal is basically 
different from extrapolation, although they use similar algorithm. The "pseudo-signal" 
in the extended domain is only used as an aid to treat or analyze the original signal in a 
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way that doesn't significantly disturb the behavior in the known region. In general, one 
does not obtain true signal in this domain. The beauty of the present periodic extension 
algorithm is that it provides a boundary condition of the signal without significant 
"aliasing", as was shown in [20] and [22] Once the periodic signal is obtained, it can be 
extended as much as desired, depending on the intended numerical application. In 
contrast, extrapolation is required to accurately predict the true fimction values in the 
extended domain. 

It must be noted that although we have discussed the algorithm in one dimension, 
extending it to two or more dimensions is straightforward. The DAF approximation to 
a function and its derivatives are given in (13) and (14), respectively. However, a direct 
two-dimensional extrapolation is a time and memory consuming procedure because too 
many simultaneous algebraic equations must be solved. One altemative to this is to 
consider the two dimensional patterns as composed of many lines of one-dimensional 
signals, and then extend each line separately. This procedure is generally less accurate 
than the direct two dimensional extrapolation algorithm because it only considers the 
correlation along one direction while neglecting cross correlation. However, for many 
problems, it is sufficiently accurate, and it is a very economical procedure. 

The algorithm presented in this section only refers to periodic extension as an 
example. However, we note that one can also extend the original signal to any other 
kinds of boundary conditions, as needed for different numerical applications. Periodic 
extension is only one case out of many possible ones. 

NUMERICAL EXAMPLES 

Numerical experiments are preformed using the "Lena image", shown in Fig. 2. 

We degrade it with Gaussian white noise with peak signal-to-noise ratio (PSNR) of 

22.14 dB [Fig. 3(a)] and 18.76 dB [Fig. 4(a)] respectively. The PSNR used here is {\it 

Idefined} to be 255 x 255 

MSB (19) 

where MSE is the mean-square-error of the noisy image, defined by 




(20) 
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where S(i j) and §(i j) are the original image and noisy image samples respectively, and 
and Ny are the number of pixels horizontally and vertically respectively. For the 
purpose of effectively removing the noise in the homogeneous, areas while 
simultaneously preserving the edges, we choose the following definition of the diffusion 
coefficient [8] 



c{s) 



s <T 
s>T 



(21) 



where ^, 7, e and p are adjustable parameters. Here we fix A=100, r=5,:€-l and p=0.5 
in our numerical applications. The anisotropic diffusion equation (1) is spatially 
discretized using the Hermite DAFs according to (14). The more "well-tempered" 
Hermite DAFs {oJA=Oy/Ay-236, M^=My=30 for the lower noise image and M^=My=22 
for the higher noise image) are used to discretize the variable of the diffusion coefficient 
to remove more noise and accurately detect the edges. A more accurate Hermite DAF 
(o_x/A^=Oy/Ay=3.54, M^=My=120) is employed to discretize the other part of the spatial 
differential operation accurately. The fourth order Runge-Kutta method [23] is employed 
to perform the time evolution. Before carrying out the time propagation, we used the 
HDAF with o^=Oy=2.36, Mjj=My=32 to remove the high firequency noise. 

Fig. 3(b) and Fig. 4(b) show the resulting restored Lena images of produced by 
our numerical algorithm. Comparing them with the degraded images [Fig7-_3(a) and Fig. 
4(a)] and the original image (Fig. 2), it is clearly seen that the algorithm can effectively 
remove noise while preserving the edges simultaneously. The PSNR is increased by 
about 8.00 dB for the lower noise image and 9.43 dB for the higher noise image. The 
boundary of the image is also effectively preserved, which shows the success of our 
periodic extension algorithm. As pointed in [9], for noisier images, the Perona-Malik 
algorithm may produce some noise echos that are larger than the threshold level which 
can not be removed. However, increasing the threshold level will lead to the 
edge-blurring problem. By using the algorithm presented in this paper, we can 
effectively avoid the problem by accurately detecting the edges using the 
"well-tempered" HDAF. The procedure of using the "well-tempered" Hermite DAF to 
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detect the edges automatically avoids having to apply an additional smoothing operation 
to the variable of the diffusion coefficient. We also test the success of our algorithm for 
even noiser images and find that it can increase the PSNR enormously («.g., we have 
increased the PSNR of the Gaussian noise degraded Lena image firom 1 1.35 dB to 23.10 
dB). Computations with the smoothing filter (6) £^)plied to the variable of the diffusion 
coeflBcient, (as presented in [9]), shows that the "well-tempered" Hennite DAF is better 
able to detect and preserve the edges than the exponential filter. 

CONCLUSIONS 

New computational schemes are disclosed to restore a noise corrupted image 
based on solving an anisotropic diffusion equation. The distributed approximating 
functional (DAF) approach introduced in this paper has several advantages for 
application to the anisotropic diffusion equation. First, it provides a periodic boundary 
condition to the image. Second, it provides a smoothing operation to the variable of the 
diffusion coefficient. Third, it discretizes the spatial derivatives of the equation as 
accurately as required for a given application. In addition, before performing the time 
evolution using the forth order Runge-Kutta method, the DAF can be used to eliminate 
the high firequency noise &om the image. Test results on a noisy Lena image show that 
it is effective for removing the noise and preserving the edges of the image 
simultaneously. It can greatly increase the PSNR of the noisy image. The resulting 
periodic extended images can be implemented v/ith many other numerical schemes which 
require a knowledge of the signal in an extended domain or prefer a given number of 
image samples. 
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TABLE I 

Comparative RBsroRATroN Results in PSNR for Image Lena 
Corrupted wrrn different amount of Gaussian noise. 



Algorithm 


Corrupted image 


22.14 dB 


18.76 dB 


Median fUter (3 x 3) 


27.24 dB 


24.72 dB 


Median filter (6 x 5) 


26.53 dB 


25.29 dB 


New approach 


30.14 dB 


28.19 dB 
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A VARIATIONAL APPROACH TO THE DIRICHLET-GABOR WAVELET-DAF 
INTRODUCTION 

Recently we have introduced several new types of distributed approximatiitg fiinctionals 
(DAFs) [1-5], and related wavelet bases associated with them [4,6-10]. In* the course of 
that work, it was observed that DAFs could be generated in several ways. \ The first 
approach [2,3] leads to a systematic way of approximating a given discrete set of input 
data with an infinitely smooth fiinction. \ The most intensively studied DAF of this type 
is called the Hermite DAF, or HDAF. The fimdamental unit of its construction is a 
product of a Hermite polynomial and its generating fimction, referenced to an origin that 
is located at each point, x, A variational method was introduced for deriving such DAFs 
[3], and they have been applied to a large number of problems, ranging fi-om solving 
various linear and nonlinear partial differential equations (PDEs) to the entire gamut of 
signal processing [6,7, 1 1 - 1 4] . A distinctive property of the first DAFs is that they are not 
interpolative on the input grid points [2,3]. That is, the first type of DAF approximation 
to the fimction at any grid point, Xj, is not exactly equal to the input data value. In place 
of the interpolative property, this DAF approach to fimctional approximation has the 
property that there are no "special points". Said another way, such DAFs deliver similar 
accuracy for approximating the fimction either on or off the grid; similarly, the DAF 
approximation to a fiinction, sampled discretely, yields an approximation to the 
derivatives of the fimction comparable in accuracy to the fimction itself This is strictly 
true only for fimctions belonging to the "DAF-class", which is that set of fiirictions whose 
Fourier transform is sufficiently contained under the DAF-window in Fotirier space [3], 
More recently, we have developed another general type of DAF which does 
interpolate on the grid, but which still can be "tuned" to yield highly accurate derivatives 
for DAF-class fimctions [4-7,11-14]. The essence of this approach is to modify an 
"interpolating shell" (such as that for Lagrange Interpolation [5], etc.) by an appropriate 
weighting fimction. By far the most attractive choice has been a Gaussian weight 
fiinction, which has the property of "regularizing" the interpolation so that it delivers an 
infinitely smooth approximation to discretely sampled fimctions [4-7], and the accuracy 
is ensured so long as the function being considered is in the DAF class. Again, these 
have been shown to be enormously robust for the class of PDEs and signal processing 
problems considered earlier [5-7, 1 1 - 1 4] . 
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An alternative way of viewing these D AF s results from observing that continuous D AFs 
constitute two-parameter Dirac delta sequences [4]. That is, they are approximate 
identity transforms that depend on two adjustable parameters. In the case, e.g., of the 
HDAFs, the two parameters are the Gaussian width, o , and the highest degree 
polynomial, M (where M is even) 

where, it is easily shown that 

Um S^^M^ - »V) = S{x - x') (2) 

for any o >0, and also that 

Urn sZk^ - ^V) = - ^0 (3) 

for and fixed M„ The availability of two independent parameters, either of which can be 
used to generate the identity kernel or Dirac delta function, can be viewed as the source 
of robustness of the DAFs as computational tools [4]. 

Of the recently introduced regularized, interpolation DAFs, a potentially very 
useful one is the Dirichlet-Gabor wavelet-DAF (DGWD) \cite{b}. \ It was constructed 
\ by combining a Gaussian with the Dirichlet kernel for generating the Fourier series of 
a function, to give 



2sm(f) (4) 



As with all of the regularized, interpolating DAFs, this product generates a scaling 
wavelet that at once is infinitely smooth and rapidly decaying in both physical and 
Fourier space [4]. The constant, Cm,o, was determined by requiring that the zero 
frequency Fourier transform, (0|o ), be normalized to imity, that is 
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^(0) = ?gSU(Ok) = £ dx6^%o{A-) = 1 
Then the "father wavelet" basis is generated by translating and scaling, so^that [4] 

(f>mnW = a 2 <l>[—^) • (6) 
A corresponding "mother wavelet" can be defined as 

Hx) = CmA^ 2sm(f) a 2sm(ff) ^ 



Because of the constraint on \|f(0), Eq(6), it is verified that t|i(x) is a "small wave", so its 
zero frequency transfonn satisfies 



^(0) = /" <is^(x) = 0 . 

J-oo 



(8) 



The computational usefulness of the DDGWD was shown by several example 
applications to the solution of differential equations [5,11-14]. A multiresolution 
analysis has been developed based on these wavelets [4,6,7]. 



In this paper we enquire as to whether these regularized interpolation DAFs can also be 
obtained in a systematic manner from the same variational principle [3] used for the 
noninterpolating DAFs; especially the HDAF [2,3]. We shall see that the DGWD can 
indeed be obtained directly from our variational principle, and the derivation bears a 
similarity to that used for the Hermite DAFs. In the next Section, we give the detailed 
derivation of the DGWD from the variational principle. 

VARIATIONAL PRINCIPLE APPLIED TO THE DIRICHLET DAF 

A general construction of the DAF approximation to a ftmction proceeds by first 
developing a suitable approximation to the ftmction at every point x in its domain. \ This 
is typically accomplished by making a basis set expansion of the form 
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(9) 



J 



Here f(x^ {\prime } |x) is an approximation to the function f(x^ {\prime } ) about the point 
X, i,e., parameterized by x. \ The quantity % B_ (j } (x^ {\prime } |x) is the j th basis function 
for the point x and % b_{j}(x) is the corresponding coefficient of this basis function for 
the expansion centered on the point x. \ The coefficients b_{j}(x) remain to be 
determined as fimctionals of the known values of f(x). A succinct expression for the DAF 
approximation can then be given by 



(although, as previously mentioned, other, more general, definitions, e.g. as 
parameterized delta sequences, are also possible [4]). To complete the definition one 
must specify how the x-dependent coefficients are to be obtained. 

There are various ways that the set of coefficients {b^ix)} can be determined. Perhaps the 
most straightforward is by the technique of "moving least squares". In this approach one 
defines a variational function X{x) for the point x of the form 



where the summation over / is over all points in the domain of x where the value of the 
function is known. (We replace the sum by an integral over all continuous regions of the 
domain where the function is known.) The quantity (x>{xi-x) is a weight function of 
arbitrary form, restricted only in that it is non-negative. For concreteness we will take 
0) to be of the Gaussian form 



(10) 



oj{xi - x)l/(x,lx) - /(Xi)!^ , 



(11) 




(12) 



where o is a parameter with units of length. It should be pointed out that, in general, the 
form of the weight can also vary as a function ofxas can the basis functions themselves 
in both type and number. (For example, we could make o vary with x, and/or choose 
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B^(x]x) from different complete sets for each distinct value of x,) \ We then write 

/(xfcix)=:E^iC«fci«)^i('') (13) 

i 

and determine the optimal values of these coefficients at a particular value of x by 
minimizing the "cost" function X{x). In general the expansion coefficients can be 
complex. In anticipation of this eventuality, we minimize the cost function with respect 
to both the coefficients and their complex conjugates to obtain 2// equations to solve for 
the real and imaginary parts of the //coefficients. This leads to the set of linear equations 

f; a;(x/ - x)B*j{xi\x)f{xi) = E{ E ^(^' ' x)B;{xi\x)Bjixi\x)}bj. , 



which can be written compactly as 

j 

by defining 

A,.= f: u;{xi-x)B;{xi\x)f{xi) 



(15) 



(16) 



and 



Cjy = E ^(^' " x)B;{xi\x)BAxi\x) . (17) 
We immediately have that 

i' 

It is important to recall that all of the quantities in this equation, are implicit functions 
ofx. 

To proceed we confine our discussion to functions on the real line and represent 
f{x]x)j our local approximation to the function centered on the point x, as a Fourier series. 
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The basis functions then are 

= -Le-^"(''-')>/^^ (19) 

where j assumes the iV^ values -{NA)l2<.j<.{N-\)ll, (Note that j takes on integer values 
for odd N and half-integer values for even N .) Here A is the grid spacing, which is 
assumed to be uniform. As a function of x\f{x'\K) is obviously periodic with a period 
domain of A^A . From Eq.(17) it is seen that £ is a kind of overlap matrix for the basis 
functions centered at x under the weight function a)(jc -;c), and due to the periodicity of 
the basis functions, we can express it solely as a function of r^mod^{x-x). As we now 
show, it is possible to invert the matrix \underIineC(r|) in closed form. However, it is an 
approximation to the inverse that, when valid, gives rise to the D AF representation of the 
function which is of interest to us here. 

It is useful to write the sum in Eq.(16) in the form 

oo oo 

I] ()i = S X^()p.ff ' (20) 

/=-oo p=-oo q 

where I'^Np+q, Here we have divided the grid into domains, each with A/" points. The 
p-sum is over all domains and the q-sum is over all points within a given domain. JVe 
take the point of origin (i,e,, p=0, q=0) to be the grid point closest to x. Then 

(JVp + g)A + i7 , (21) 
where -A/2ii\^A/2. That is X+J7 is the grid point closest to x. \ This leads to 



J 00 N-l 

N. 



p=— 00 17=0 

N-l 



(22) 



9=0 



where 
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oo 

V.M = E u{[Np + g + „/A]A) ^^^^ 



P=-oo 



and 



(24) 



The quantity i|;j(q+T|/A) can be taken as the jth component of an orthonormal basis set 
of //-vectors indexed on 9. That is 



(N-\)/2 

(9+T,/A)* ,(9+fj/A) __ r 
i=-(N-l)/2 (^•'J 



which is a standard result from Fourier theory. \ From this point of view, Eq.(\ref {eq 13}) 
simply gives an expression for the C_{jj^{\prime }} matrix element of 
\underIine{\underline{C}}(\eta ) in its spectral representation. (Here we have indicated 
explicitly that the matrix is a function of \eta .) \ From this it follows immediately that 



_ T n "'"'Vr '^"^ (26) 



and, further, from Eq.( 1 8) that 
i' 



The sum over j' here produces the Kronecker 6^^., where ; 
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Summing over q then leads to 



^ (29) 



which is the desired variational expression for the expansion coefficients. Finally, from 
Eqs.(lO), (13) and (19) we have that 

1 (Af-l)/2 

foAFix) = ^ bjir,) 



_ ^ 1 1 (N-l)/2 

= izL)'' f sin(7r7?/A) 



which is the fomial-DAF expression without approximation. 

If f(x) is periodic with period NA, then/(;c,.) depends on only q* (z.e., not on p*) 
and the final result of Eq.(30) reduces to 

= g-)^l]7g^'^<''--'-)' (3.) 

where we have made use of Eqs.(21) and (23), (Recall that aH-ti is the grid point closest 
to X.) This is just the standard Fourier approximation to a periodic function known on 
N equally spaced grid points. It is inteipolative (i^e,Jj^^(x^)=J{x^)) where x, is any grid 
point and hence for which t] =0. This is, of course, the anticipated result for a 
least-squares fit of a periodic function using a Fourier basis. 

lff{x) is not periodic then /daf(^) is nowhere exact (unless accidentally so), and, 
in particular, the DAF approximation is not interpolative. The quantity y(9 'A +t| +;c) in 
Eq.(3 1 ) is replaced by 



7 _ a^(xr--z) 

p'=-oo '^fl'+ry/A 



(32) 
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which is a weighted average across the infinite grid of functional values on grid points 
separated by multiples of NA . 

It is clear that the DAF approximation of Eq.(30) (being basically, a Fourier sum) 
suffers from the principal drawback of the Fourier representation, namely that the 
approximation is not tightly banded. That is (off the grid) all of the N values of "f^ 
contribute more or less eqtiivalently to the approximation. Said another way, each grid 
point contributes through the normalized probability co(j:,-x)/X(j+^^, which falls off much 
more slowly than o)(x,-x) itself as q is varied. To introduce a more tightly banded DAF 
representation of the function, we now assume that \underline Q{r]) can be effectively 
replaced by a matrix that is independent of r\. In so doing we, of course, ignore 
variations in C(r|) over the distance of the grid spacing. There are various ways that this 
can be done. In previous studies, where we employed a polynomial basis set rather than 
the circular functions of Eq.(19), it proved convenient to replace Q(t]) by its average. 
This allowed us to use the properties of orthogonal polynomials to construct the 
corresponding approximation to C^ We referred to the resulting representation of the 
function as well-tempered because it has the property that for functions where the 
approximation is applicable (so-called DAF-class functions) the fit is of comparable 
accuracy both on and off the grid. In contrast, in the present case it is convenient to make 
an T|-independent approximation to C(r]) for which the grid points are special. 

The idea is that as N becomes large and the grid spacing becomes small in such 
a way that NXDelta is held constant, every point becomes effectively -a grid point 
(assuming continuity of the function to be fit). Then, to controllable accuracy we can 
replace Q(r\) by £(t]=0) to obtain 

g (33) 



(34) 



6i 



and 
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foAFix) « - u{x, - x)-j= x: v-r'^^'Vcxr) ^^^^ 

^ oo (W-l)/2 
'^0-^ «'=-eo i'=-(N-l)/2 

where X=i\^A. The applicability of this approximation depends of course on the 
appropriate choice of the DAF parameters, which has been discussed elsewhere. \ The 
sum can be written in terms of the Mth order Dirichlet kemel, Df/y), defined by 

-«W = i(i-g-'(%)) = =^^, (36) 

which leads to the expression 

, , 27rA ^ . /M^:/'-^;) . 

V^,,^ a;(xr - x)25n^(— ^--^^ )f{xv) . ^3^^ 

This result is parameterized by three quantities L, N (which are related by the grid 
spacing A) and a (see the form of co of Eq.(12). 

Since our approximation C(r]) = £(ti=0) is exact on the grid, this approximation 
is interpolative for functions that are periodic on a domain of length L. \ If we take the 
limit N -00 and L -ooin such a way that A=L/Nis fixed, then the approximation assumes 
the sinc-DAF form 

W(x) = - x) , (38) 

where we have used the fact that Xq^I in this limit. This 
resuh is interpolative on all grid points. 



CONCLUSION 

We have shown that the variational principle aised earlier for generating 
noninterpolating DAFs (which could be used to generate associated wavelets) can also 
be used to derive interpolating DAFs, with a Gaussian weight, that were first obtained 
by multiplying various interpolation shells with a Gaussian, which regularized the. 
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function (making it infinitely differentiable) and ensured that it decays rapidly both in 
physical and Fourier space. We therefore conclude that the interpolating and 
noninterpolating DAFs are very closely related, corresponding to different ways of 
solving the moving least squares variational algebraic equations. This result complements 
the earlier procedure used to construct the interpolating DAFs and provides another 
framework in which to develop robust approximation and estimation algorithms. Both 
the interpolating and noninterpolating DAFs, of course, have been shown previously to 
be computationally robust. [1-2,6-7,1 1-16] 
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WAVELETS. DELTA SEQUENCES AND DAFs 
INTRODUCTION 

The rapid development and great success of wavelet theory and technology in the 
last decade [1-7] have stimulated intense interest among mathematicians, engineers, 
physicists and chemists. New results are regularly reported and applications are found 
in virtually every discipline of science and engineering [8-10]. The basic theory and 
construction procedures are regarded as well imderstood. However, there are areas where 
the existing wavelet methods encounter difficulties. One such area is computational fluid 
dynamics and more generally computational chemistry and physics, and in mechanics. 
Meyer [4] has recently posed the questions "Can wavelets play a part inthe study or the 
understanding of the Navier-Stokes equations?" He concluded that **we still do.not know 
the answer to this question". Since wavelets are intimately and significantly related to 
spline theory and the theorj^ of approximations, it is likely that wavelet theory can lead 
to entirely new approaches for scientific and engineering computations, where traditional 
method are either global or local. It is well known that global spectral methods are 
accurate and efficient for linear partial differential equations (PDEs), whereas local 
methods are simple and convenient for nonlinear PDEs. It is extremely important to 
develop an approach that delivers global method accuracy, while also providing local 
method flexibility and simplicity, for nonlinear PDEs involving singularities, or 
homoclinic orbits, for which obtaining accurate and stable numerical solutions is still a 
major challenge [11,12]. Wavelet theory has been intensively studied for this -purpose 
[13-23]. 

The rapid growth and unprecedented success of wavelets have led most recent 
researchers to focus more on the exploration of new applications rather than reflecting 
on the basic concepts. The predominant view of wavelets has been strongly influenced 
by the belief that there are analysis tools for describing a signal efficiently in time and 
in frequency simultaneously, thus overcoming the classical linutation ofFourier analysis, 
which is strictly efficient either in the time or the frequency domain. As pointed but by 
Flandrin and Goncalves [24], wavelets and wavelet transforms are basically time-scale 
tools, rather than time- frequency ones. Moreover, as far as time-frequency analysis is 
concerned, there are many other approaches which are able to outperform both Fourier . 
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analysis and wavelets in certain cases. There are other views, which are somewhat less 
influential and less developed. For example, Holschneider has recognized that a real- 
valued, non-negative scaling wavelet provides a smoothed version of a function /over 
the real line R by means of the convoluation product 

$a.b(/) =< Mb), f >= dt^<f> {^) /(0VC(6) * /. (1) 

Then a wavelet transform given by 

Where i|i(x)==(x5+l)<l)(x) and i|;„(x/a)/a, provides a mathematical microscopy of 
/ at length scale a. These are important ideas and they deserve additional study. 

In a separate development, Hoffman, Kouri and co-workers [23-29] have 
presented a powerful computational method based on distributed approximating 
functionals (DAFs), for various numerical applications, including solving linear [30] and 
nonlinear [31] partial differential equations (PDEs), signal analysis and the padding of 
experimental data. On the real line R, the DAFs are multiparameter delta sequences of 
the Dirichlet type, constructed using functions of the Schwartz class. The success of both 
wavelets and DAF's in a variety of applications has led to a search for a connection 
between them. Indeed, there is a natural connection between the [32] ^d numerous 
wavelet-DAFs and DAF-wavelets have been discovered [32,33]. Wavelet-DAFs have 
been used to obtain results in solving the Navier-Stokes equations [34] with nonperiodic 
boundary conditions. In fact, underlying both wavelets and DAFs is a common 
mathematical structure, the theory of distributions, which was initiated by physicists and 
engineers, and was later presented in a rigorous mathematical for by Schwartz [35], 
Korevaar [36] and others. A general distribution analysis of wavelets has been given by 
Meyer [1], Meyer and Coifinan [4]. Daubechies [3] and others [37]. Orthogonal wavelet 
expansions of the delta distribution are discussed by Walter [38]. . Howev^, the role of 
delta sequences in wavelet theory has hardly been addressed. In particular, -a l^e class 
of delta sequences can be identified as scale wavelets, and they can be used as muUi- 
resolution wavelet bases, which we call delta-sequence-generated multi-resolutioa 
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wavelet bases (or DAF-wavelets). The purpose of the present work is to provide a 
unified description of wavelets, delta sequences and DAFs. 

REVIEW OF DELTA SEQUENCES 

The deha distribution or so-called Dirac delta function began with the Heaviside 
calculus and was informally used by physicists and engineers before Sobolev, Schwartz 
[35], Korevaar [36] and others put it into a rigorous mathematical form. There are 
several formal mathematical constructions which have been used in the literature (35,36). 
We shall take the approach which obtained the delta function as a sequence limit (2). 

A sequence of functions djx) e -^i 4)(jc), is a delta sequence oiii the domain/ if 
for each xe J and suitable functions 



where the sequences are generalizations of Cauchy sequences and are called 
fundamental families on J by Korevaar [36], 

Two types of delta sequences (the positive type and the Dirichlet type) can be 
used as the basis of a useful classification scheme. Both types are discussed in the 
following two subsections, (Moreover, classifying delta sequences according to whether 
they belong to the Schwartz class or non-Schwartz class is also very useful for various 
physical and engineering applications.) 

Delta Sequences of the Positive Type 

Let {3J be a sequence of functions on (-«,«») which are integrable over every 
bounded interval. A delta sequence { <5„} is of positive type if, for any bounded interval 
containing the origin, the integral in the a-^a^, limit of the sequence of the functions 
equals 1. For bounded intervals excluding the origin, the integral of the sequence of 
functions equals zero in this limit. All members of the- sequence are positive semi- 
definite. To illustrate, we consider first the example of a delta sequence of-impulse 
function. 

To approximate idealized physical concepts such as the force density of a imit . 




(3) 
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force at the origin jc = 0, or a unit impulse at time x = 0, one has 



/ 




a for 0 < X < 1/a a = 1,2, 



(4) 



0 otherwise 



as a delta sequence in the limit a— 

A second example is Gauss' delta sequence, given by 



(5) 



and it arises naturally as a distribution solution or so-called weak solutionis Gauss' delta 
sequence has various interesting properties with regard to differentiability, boiindedness 
and Fourier transforms, and is used to generate the "Mexican hat" wavelet. Various other 
wavelets can also be generated from it as described in later sections of this paper. 
A third example is Lorentz's delta sequence. 



which is used in representing the solution of Laplace's equation in the upper half plane. 
It is also commonly seen in integral equations involving the Green's function of the 
kinetic energy operator (in the momentum representation). 



A systematic procedure for generating various delta sequences of thcpositi ve type 
is a follows: 




(6) 



Other examples included Landau's, Poisson's and Fejer's delta sequenees. 



(7) 



Delta Sequences of the Dirichlet Type 

Definition 4. Let {6a} be a sequence of functions o (-<», «») v/hich — every 
bounded internal. We call {6a} a delta sequence of the Dirichlet 



L 



J 6a - 1 as a - ttQ For some finite constant a. 
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2. For every constant 7 > 0, + XT) 5a 0 as a 

3. There are positive constants - and - such that 

i a: I 



ao. 



for all X and a. 

Example L Dirichlet's delta sequence 

An important example of a delta sequence of the Dirichlet type is Dirichlet ' s delta 
sequence 



5a{x) = < 



Z)a(x) for|a;|<7r for a = 0,1, 2,- 
0 otherwise 



(1-7) 



where D„ is the Dirichlet kernel given by Eq. (12). Dirichlet's delta sequence plays an 
important role in approximation theory and is the key element in trigonometric 
polynomial approximations. 

Example 2. Modified Dirichlet 's delta sequence 
Sometimes there is a slight advantage in taking the last term in D„ with a factor of 14: 

^;W = I>a-^cos(ax) 
sin(aa;) 

-2:rtan(H' ^ = ^'2' ' ' " ' (18) 



This is the so-called modified Dirichlet kernel. The difference - D„* tends uniformly 
to zero on (-tc, tt) as a <». They are equivalent with respect to convergence. The 
expression given by 

r 

for|x|<7r for a = 0,1, 2,--- 
0 otherwise 

is a delta sequence of Dirichlet type as a -* «>. 

Example 3. Dirichlet 's continuous delta sequence 
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Dirichlet's continuous delta sequence is given by the following Fourier transform of the 
characteristic function xi'^^^V 




sin(aa) (20) 

TTX 



This converges to the delta distribution as a - ~. Eq. (20) is related to Shannon's 
sampling theorem in information theory. 

Example 4, The de la Vallee Poussin delta sequence 
The de la Vallee Poussin delta sequence is given by the Fourier inverse of the following 
function 



It is easy to show that 



^ otherwise 



: <iei<2^ • (21) 



SJx) = - gos(Q=g) - cos(2ag) 

^ ' Tca ^2 (22) 



is a delta sequence of the Dirichlet type as a - «. 

Example 5. Interpolative delta sequence 
Let {6„} be a sequence and let {xJo" be « + 1 zeroes of a Jacobi polynomial in (a, b). 

> 

is a delta sequence as n This follows from the fact that jAJx,y)/(y)dy 

approximations to the Lagrange interpolating formulae which converges as n ^ «> and sup 



Example 6. Delta sequences constructed by orthogonal basis expansions 
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Let be a complete orthonormal Z.^(a,b) basis. Then 

n 

Sn{x,y) = M^)<t>iiy)^ y e (a, b) (24) 

are delta sequences. In case of trigonometric functions, we again obtain the Dirichlet 
kernel delta sequences given in the Examples 1 and 3. A Hermite function expansion is 
given by 

i„W = =.p(4)|;(^)'-^ir.(^). v,e« (25) 

where H2i,iTi/y/2) is the usual Hermite polynomial. This delta sequence was studied by 
Schwartz [44], Korevaar [36] and was independently rediscovered by Hoffinan, Kouri 
and coworkers [25] in a more general form. Various other cases can be found in Walter 
and Blum's reference [39] and these also have been studied in very general forms by 
Hoffinan, Kouri and coworkers [28]. 

WAVELETS AND DELTA SEQUENCES 

Wavelets have been widely used as an analysis tool for various applications. The 
essential reason for this is because both orthogonal and nonorthogonal wavelets can 
provide a decomposition of a function at a variety of different scales. In other words, 
wavelets form special!^ (R) bases or frames for representing a function at various levels 
of detail, leading to so-called mathematical microscopies. This turns, out to be very 
efficient for approximating and analyzing functions in many applications. Orthogonal 
wavelets and multiresolution analysis have been successfully used in a variety of 
telecommunication and engineering fields [10]. They play a special role in those 
applications where orthogonality is strongly required. In many other applications, 
nonorthogonal wavelets, or frames, are also very useful. Generating new types of 
wavelets has been of great importance in wavelet theory. Most of the delta sequences 
described in Section II can be regarded as scaling or father wavelets. These father 
wavelets can also be systematically transformed into mother wavelets. The orthogonal 
wavelets are briefly reviewed in the first subsection. The connection between delta 
sequences and wavelets is made in the second subsection. The construction of mother. 
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wavelets from various delta sequences is discussed in the last two subsections. 

Orthogonal Wavelets 

The formal theory of orthogonal wavelets on L^(R) has been presented in many 
books [1-3]. An orthogonal wavelets system is usual generated by a single fiinction, 

either a father wavelet (porz mother wavelet ^, by a standard translation and dilation 

technique 

<t>mn{x) = - , m,n € Z\ (26) 

i)mn{x) = _ n) , Tn,n G Z, (27) 

where the symbol Z denotes the set of all integers. This can be formulated rigorously in 
terms of a multiresolution analysis, i.e., anested sequence of subspaces {V„J^ m E Z^such 
that 

1. {^(x — n)} is an orthogonal basis of Vq\ 

2. •••CViCVbcVliC--'CL'(i?); 

3. f[x)EVm^ f{2x)^Vm-u 

4. n„,V;„ = {0} and u;;^^ = L\R). 

Since (f)EV^ c F,, it can be expressed as superposition of { „} which constitute 
basis for F , 

<^W = E^n^l.n, (28). . 

where is a set of finite coefficients. 

For an orthogonal system, the subspace F , can be further decomposed into its 
orthogonal projection in and a complement 

where is a subspace spanned by orthogonal mother wavelets { J^) . In general, ij/^;^^, 
neZ is an orthogonal basis of W,^ and 
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© = L'iR). (30) 



r2 



It follows that {m,nEZ)ism orthogonal basis of (/?). Similarly to Eq. (28), 
the mother wavelet can also be expressed as a superposition of { „} - 

where6„ = (-ira,.„. 

Perhaps the simplest example is Haar's wavelet system [2] which is given by 
^x)=X[o.ijW> characteristic function of the interval [0,1]. It obvio.iisiy has 
orthogonal translations. The dilation of ^ (x) results in characteristic functions for 
smaller (or larger) intervals and each of them spans a subspace by translations. 

It is not obvious that a multiresolution analysis exists for ^ other than the 

Haar system. The construction of the first few orthogonal wavelet bases was more or 
less an art rather than a systematic procedure; it required ingenuity, special tricks and 
subtle computations. One procedure, due to Meyer [1], is to begin with a spline 
function 6(x)=(l-|x-ll)X[o,2) which, by translations, generates a nonorthogonal Riesz 
basis (a frame of the lease possible redundance. The corresponding orthogonal basis 

{ ^} is represented in Fourier space as [add from text] 

where a(^) is function to be determined. Using both the orthonormality requirement, 



< <^o,o,^o.n >, (32) 



and the periodicity, ^can be resolved as 

Daubechies presents another approach for constructing orthogonal wavelets. 
In the Fourier representation, the dilation equation can be written as 

^(0 = moU W(^/2), ^^^^ 
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where /Wq is a 27r-periodic function. The orthonormaUty condition then requires 

I Tno(e/2) p + I moU/2 + tt) p= L (35) 

It turns out that if the set of expansion coefficients a„ of Eq. (28) are chosen as 
ao=v(v.l)/(v-M)y2, a,= .(v-l)/(v+l)x/2, a^ = (v-l)/(v+l)v/2, a3= v(v+l)/(v+l)y2 (veR), 

then Eq. (35) will be satisfied and consequently ^can be fcfund recursively. 

Delta Sequences as Father Wavelets 

Let {6„} where a a^htz, sequence of C7" functions on ( -co, oo) which are 

integrable over every bounded interval and 

4(0) = 1 for each a; 

3. for every constant 7 > 0, (/li + /-T) ^'=* ^ ° ^ " 

4. and || x8a{x) ljoo< 00 for all i and a. 

Then the {6„} are delta sequences and each function can be admitted as a father wavelet. 
We call this class of father wavelets "delta-sequence-generated father wavelets 4)^". The 
corresponding "delta-sequence-generated mother wavelets have the Fourier 
transform property 

^a(0) = /^a(x)dx = 0. [36) 

It is natural to view delta sequences as father wavelets. In particular, if the delta 
sequence has the structure that 6a=(l/a)p(^/a) and Jp(x)dx=l as is the case for many 
examples given in Section II), then 6„ is a sequence of father wavelets at different scales. 
In contrast to the delta distribution which has only a point support, a function in a delta 

sequence can have an arbitrary support, depending on the scale. In the limit the 

delta sequence converges to the delta distribution and the support shrinks down to a 
point. The resulting delta distribution actually helps' to fiimish the" wavelet 
multiresolution analysis [37] 
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. {S}®^Wm^L'{R), (37) 

where {6} is the space spanned by the delta distribution. This is in contrast to Eq. (30). 
Clearly, if a delta sequence is an orthogonal system, such as Dirichlet's continuous delta 

sequence, for a fixed at ^ a^.v^t have 

{8a} e 0V„, = {8} ® © = L\Rl (38) 

where {6„} spans the wavelet subspace Vq. Hence the orthogonal delta sequence spans 
the wavelet subspace {6}0® " for an appropriate choice of a. 

Delta sequence generated mother wavelets can be constructed in rnany different 
ways. We discuss two approaches in the next two subsections. 
Wavelets Generated by Differentiation Pairs 

For a given C" father wavelet cj), we define a family of wavelet generators 

"^5^+^5^> n = 0,1,2,. ..,m (39) 
for generating a family of /m + 1 mother wavelets 

Va.„(a=) = G^"Va(x) for (^eC- and n = 0,1,2,..., m. (40) 

It is noted that this approach is not restricted to the delta-sequence-generated:wavelet and 
is actually a very general and efficient way for creating wavelets from a given C" father 
wavelet. The transform prescribed by Holschneider [6], Eq. (2), is a special case of our 
family of wavelet generators. 

Our wavelet generators are closely related top the transformation Lie group of 
translations and dilations. This is because the Fourier images of distributions 

^ = ,|l+„|^ „ = 0,1,2,- ■ («) 



form an infinite C" dimensional wavelet Lie algebra with elements {Jf„=/'*^|n=0,l,2, 
. . .} (here we follow the convention that statements concerning the structure of a Lie 
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algebra are made only on the basis of the real Lie algebra). The whole Lie Algebra 
structure of {X„} is simply given by 

[Xn,Xm]^{m-n)Xm+n-2 71,7X1 = 0,1,2,-... (42) 



generates a one-parameter non-compact abelian group which is obviously the 
translation group in momentum space. X2 generates a dilation group. There are two 
nontrivial invariant subalgebras 

[XuX2] = Xy; and (43) 
[Xi,X2] = ^1, [^^1,^3] = 2^2, [X2,X^] = X,. (44) 

X^,X2 are respectively the infinitesimal generators of a two-dimensional translation and 
dilation group. The third element is a quadratic dilation (superdilation), which allows 
us to generate another invariant subalgebra, Eq. (44). This result indicates that our 
method of systematically generating wavelets is very general and has a mathematically 
rigorous foundation. More details will be presented elsewhere [45]. ^ 

Example J. Mexican hat wavelet and generalizations j ^^/"P^ 

If we take Gauss's delta sequence as a father wavelet ^^^^^i~z'7?^ ^ 
thenEq. (40), for « = 1, is 



which is the well-known Mexican hat wavelet [3]. Taking w = 3 yields 



which is an interesting "Mexican superhat wavelet". This wavelet js expected to perform 
better than the Mexican hat for some application. Since elements of Gauss* delta 
sequence are functions, there are infinitely many Gauss-delta-sequence-generated 
wavelets given by 
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'^'""^''^ - ^ Vl^~2-^»-(vI^j« n = 0.1,2.... 



(47) 



It is seen that the celebrated Mexican hat wavelet [3] is just a special case of Eq. (47). 

It is interesting to note that all higher order Hermite functions (h'^O) are mother 
wavelets, while the lowest order Hemite function is a father wavelet. This can be 
naturally seen from the orthonormality condition 

y^aV7r2"+-+in!m!^" (vla) (vla) "'^"^'^ " n,m = 0, 1,2, ... . (48) 

Here, if the second polynomial if fixed as m = 0 to a given constant, then the only case 
to to give a nonzero integration is w = 0 which determines a father wavelet. All other 
Hj[n^O) give rise to mother wavelets. We point out that this result is not limited to 
Hermite polynomial. It is generally true for all polynomial systems which form 
orthogonal L^(a, b) bases with respect to an appropriate weight. More details about the 
general connection between wavelets and conventional Hilbert space bases will be given 
elsewhere [46]. 

Example 2. Shannon 's wavelet family 

Dirichlet's continuous delta sequence is related to the well-known Shannon's 
father wavelet ^1^)- tt x' The latter is known for generating an orthogonal 
basis for a reproducing kernel Hilbert space. A family of (mother) wavelets can be 
generated by using our wavelet generations, Eq. (39) 

sin(ax) 



= -sin(ax) for n = 0 

It 

—a 

= cos(ax) for ti = 1 

TT 

= — sin(ax) for n = 2 

TT 
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— L sin(ai) for n = 2g 

= ^^J±^ cos(ax) for n = 2g + 1. (49) 

X 

± ^. 

These results are in contract to Shannon's wavelet, 1T](\^^^'^^^^^^^^^'] 
Obviously, all of these wavelets can be used to generate orthogonal wavelet bases by the 
standard method of translations and dilations. It follows that if the starting father wavelet 
generates an orthogonal system, then, the corresponding wavelets created by our wavelet 
generators, Eq. (39), are also orthogonal systems. 

It is very easy to constmct various delta-sequence-generated wavelets by applying 
examples given in Section U to the right hand side of Eq. (40). 
Wavelets Generated by Difference Pairs 

Another simple and efficient way of generating wavelets from delta sequences is 
to take the difference between two normalized elements in a delta sequence 



Example L Hermite wavelets and the Mexican hat wavelet 
In case of Hermite' s delta sequence, Eq. (25), we have 



V'n.n' = e 



E (t) (52) 



This is a general expression for a family of nonorthogonal wavelets. In particular, if n 
= r=0, we obtain 

V'i.o(x) = ^(l-«^)e^. (53) 

This is, once again, the well-known Mexican Hat Wavelet [3]. The Hemiite wavelets 
described in Eq. (47) can easily obtained, within a constant difference, by appropriately 
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choosing n' = n - 1 in Eq. (51). 

Example 2. Shannon *s wavelet 

We can use Dirichlet's continuous delta sequence as a father wavelet 
_. Then the corresponding mother >Yavelfetg generated by 

Eq. (50) are 

^ for a//3#0. (54) 

This family includes the well-known Shannon's wavelet [3] as a special case 

i^2irA^) = — [sin(27rx) - sin(7rx)] . Y55) 

TTX ^ ' 

It is easy to check that this Shannon's wavelet generates an orthogonal system. Equation 
(55) is in contrast to the other family of Shannon's wavelets, Eq. (49), produced using 
our wavelet generators. 

Examples, Gauss ' wavelets 

It is noted that this procedure of generating wavelets is also very general. For 
example a wavelet can be constructed by combining a pair of functions from the Gauss' 
delta sequence, 

Va^(x) = -^e-V2.' _ -i^e-'/^''' for aj^fi^O . . (56) 

Note that this is not a special case of Example L 

In the case where there is more than one delta parameter, the corresponding 

wavelets can be generated as differences of cross terms. This will be discussed 

in detail in Section V. 
DISTRIBUTED APPROXIMATING FUNCTIONALS 

Definition 5, A fimction is said to be of the Schwartz class if it is a C*^ function 
and rapidly decaying, 

sup|x'*a^'^,(x)|<oo, >S7) 
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for a^O and all pairs of / and y. The set of all functions of the Schwartz class is denoted 
by S. 

Definition 6, Distributed Approximating Functionals (DAFs) .are families of 
functions which satisfy the following: 

1 . They are sequences of Schwartz class functions on the real line R; 

2 . They are multiple parameter sequences and each parameter independently 
leads to a sequence of functions which converges to the delta distribution 
6 

3. They are delta sequences of the Dirichlet type. 

Definition 7. A function on ( - ^) is said to be of the DAF class if it is 

integrable over every bounded interval and is of at most polynomial grov^ 
asymptotically. 

Fourier transforms of Schwartz class functions are still Schwartz class functions. 
DAFs are smooth and rapidly decaying functions in both Fourier space and ordinary 
space. We note that these properties are crucial to the usefulness of DAFs in various 
numerical applications. DAFs are a multi-parameter system. The best computational 
efficiency can only be obtained in certain regions of the parameter space. For Hermite 
DAFs, we call these well-tempered regions. (Note that well-tempered used here has 
nothing to do with the tempered distributions which arise naturally in the theory of 
Fourier transforms.) By well-tempered, we mean that the DAF approximations to a DAF 
class function are of the same level of accuracy both on and off a grid or DAF 
approximations of the derivatives of the function have compatible accuracy to that of the 
function. By compatible accuracy, we mean that the accuracy decreases less than a factor 
of two as the order of differentiation increases by one. (Note that pointwise 
differentiability is not required for the DAF class functions because generalized 
derivatives exist for the DAF convolution of a DAF class function. Certainly generalized 
derivatives are, in general, not necessarily functions but are distributions. That is how 
DAFs can lead to distribution solutions to a partial differential equations, and why they 
turn out to be extremely powerful for approximating functions which have discontinuities 
and even singularities on a set of measure zero.) An alternative way of understanding the 
well-tempered region of DAF parameter space is to view it from the wavelet 
multiresolution analysis point of view. Essentially, there is an interplay of three factors: 
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(i) the DAF expansion of order M (which determines a father wavelet), (ii) a DAF 
window size a (which determines the scale or correlations), (iii) the DAF central 

frequency ^ 1/A (which is equivalent to a dilation parameter). The best computational 

efficiency is achieved when, for a given M, the DAF window size is proportional to the 
central jfrequency, a«A. 

The last condition in our definition of DAFs reflects the fact that the most 
important DAFs we have discovered so far are delta sequences of the Dirichlet type. 
From the point of view of approximation theory, delta sequences of the Dirichlet type 
are, in general, more rapidly ohvergent than those of the positive type [39], However, 
for providing imbiased approximations, the delta sequence of the Dirichlet type is 
applicable to a smaller class of functions than that of the positive type. Specifically, the 
function is required to be at lease Holder-continuous or C\ This requirement is not 
needed for the DAF approximator because DAFs are of Schwartz class. An additional 
benefit of DAFs being of Dirichlet type is that DAFs are indeed "small waves" [2] and 
are readily admitted as father wavelets. Corresponding DAF mother wavelets can be 
generated by the techniques described in the last section. 

In parallel to the orthogonal wavelets and nonorthogonal wavelets, there are two 
classes of DAFs: orthogonal DAFs which are generated by orthogonal basis expansion 
of the delta distribution and nonorthogonal DAFs which are created by methods of 
regularization. These are discussed in the following two subsections. It is noted that 
both orthogonal DAFs and nonorthogonal DAFs are firames, rather than orthogonal father 
wavelets. 

A. DAFs Generated from Orthogonal Systems 
One of the most important DAFs is the Hermite DAF (HDAF). It was first 
introduced in a discrete fi-om [25] by Hoffinan, Kouri and coworkers for quantum 
dynamics and then in a continuous form, by Kouri, Hof&nan and coworkers [26], as 

W(x - x' 1 M.<r) = i exp E (t) . -^^''^ (^) ' 

where f"^?^^) are the usual even Hermite polynomials. While the 

Hermite expansion of the delta distribution, Eq. (25), was discussed by Schwartz [35] and 
Korevaar [36], the Hermite DAF is very different, at least where applications are ' 
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concerned, because it is explicitly a two-parameter delta sequence. This has the 
consequence that the Hermite DAF %ffnApC)C - IVl^V^ converges to the delta 
distribution either when the degree of the polynomial M goes to infinity. 

i^Zo - I ^, cr) = 6{x - x') (59) 

independent of o^O or in the limit o -> 

Um Shdaf{x - x' I Af , a) = 5{x - x') (60) 

independent of 0 < M< Having two independent DAF parameters is cnicial to its 

success in various numerical applications, because adjusting two DAF parameters leads 
to tunable accuracy within well-tempered regions. Without the introduction of the 
dilation parameter o in the Hermite expansion of Delta distribution, Eq. (25) itself would 
have very little numerical utility. Well-tempered Hermite DAFs have been used for a 
variety of numerical applications, including among others, filtering and fitting 
experimental data and ab initio quantum mechanical potential surface values, padding 
data on two- and three- dimensional surfaces, resolving eigenvalues of a Hamiltonian 
solving linear and nonlinear PDEs, and signal processing. Some of these applications 
involve estimating the derivatives of a fimction known only on a finitej discrete grid, 
which are given in our DAF method in the sense of a distribution [25], 

= < -x'\M, cr), /(x') > (61) 

= (-1)' < Shdaf{x - x' 1 M,<r), /(')(!') > , (621 

where S]§\p{x - x' | M,<r) is given by [25] 

(63) 

As stated earlier, the Hermite DAF is a Schwartz class fiinction for non-zero o and the 
finite Af, the right hand side of Eq. (61) exists in a distribution sense of/^ is not well 
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defined. This situation can occur in solving PDEs for which Hermite DAFs can smooth 
out the discontinuities and "round off' singularities so as to provide numerical solutions 
(so called "weak solutions")- This is the case in solving the Fokker-Planck equation 
when the initial-distribution function /is a Dirac delta distribution.^ Hermite DAFs have 
successfully estimated the first and second derivatives and integrated the Fokker-Planck 
equation to an error of the order of 10"'^ [30]. 

The Fourier space representation of the Hermite DAF is given by [27] 

w«iM..)=«p(-«^)|m. m 

This has the important property that 




Hence, Hermite DAFs are readily admissible as father wavelets. In parallel with Eqs. 
(59) and (60), it is easy to see that Hermite DAFs become a Fourier space all pass filter, 

A 

i.e., £uo(icCf^ 1 lV^^"^-l whenever -* ^ or a For a given set of Hermite 

DAF parameters ^ f «^ /^9^^5^^Dfrg(^j|lW^?>) is an infinitely smooth, exponentially 
decaying low pass filter which is extremely useful for signal analysis. 

DAFs of Fejer type can be easily generated by using our Hermite. DAFs 

Shdaf^FcjUx - I M, (t) = exp [—^^ ) 

A somewhat more general approach is to sum selectively 

2 {-{x-x'yX 

SHDAF.Fejcr{x " x' |,M, M',Or) = ^^j^ ^ J^, y^ ^^ ^ 2^ j 



We point out that since DAFs are multiparameter delta sequences, averaging over 
other DAF parameters will also generate DAFs of Fejer type. We shall not, 
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however, give an exhaustive list of these possibilities here. 

A particular, special case of our Hermite DAF (for Af = 2) has been 
previously discussed by Monaghan and Gingold [47} in terras of a super Gaussian 
kernel, 



(Note that this is different from the Mexican hat wavelet.) These authors noted 



integrations and performs better than the Gaussian kernel. 

More general DAFs based on orthogonal basis expansions can be 
constructed in two ways, as described in Ref. [28]. Essentially, for a given 
positively defined weight function w in a domain (a, b), it is possible, by using the 
standard three term recurrence method, to generate a set of polynomials P„ which 
is orthonormal under the inner product with respect to the weight w. This results 
in very general orthonormal bases. All classical polynomial systems, such 

as Hermite, Legendre, Jacobi, Chebyshev, etc. are special cases of this approach. 
This procedure of generating arbitrary orthonormal systems is well-known and has 
been employed by Shizgal [48] for constructing various reproducing kernels, 
which, when limited to the classical weight function are coincident with the 
discrete variable representation (DVR) [49} . If the weight function of a general 
orthonormal basis is chosen to be of the Schwartz class a general DAF can then 
be constmcted by expanding the delta distribution in the domain (a,b). The 
resulting DAF is an approximate identity kernel and is very similar to the 
reproducing kernel of Shizgal for a common weight w. However, there are 
important differences in the philosophies of the two approaches. The grid points 
in the Shizgal' s method are always restricted to the whole set of nodes of the 
polynomial of highest degree occurring in the expansion. As a result, both the 
function and its derivatives are approximated in a global maimer, necessarily 
involving all grid points in the domain (a,b), A major advantage in the DAF 




(68) 



that this super Gaussian kernel gives interpolation error of 0{a^) in numerical 
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approach is that there is no restriction on grid selection. It turns out that because 
of the nature of the Schwartz class DAFs, numerically, the DAF approximation to 
a function and its derivative at each point jc is effectively a localized one, with 
contributions coming only from near neighbor points, x'. - The range of the 
neighborhood is controlled by the DAF window parameter, o in case of Hermite 
DAFs. The functional relation determining the contributions of neighboring grid 
points is given by a pointwise basis set expansion of the delta distribution and 
controlled by another independent DAF parameter. Therefore, the DAF approach 
is a pointwise spectral method and its matix representation is banded. It is well- 
known that, in general, global methods are more accurate than local methods, 
while local methods are flexible for handling complex boundaries and geometries. 
Therefore, linear systems with simple boundary conditions are preferably solved 
by global methods, while nonlinear systems with complex boundaries are usually 
solved by local methods. However, in a wide range of numerical applications, 
such as long-term weather predicting, describing shock waves in compressible gas 
flow, or vortex sheets in high Reynolds number incompressible fluid flow, and 
studying critical points of Bose-Einstein condensations, one deals with nonlinear 
PDEs possessing singularities or phase space homoclinic orbits [11,12], The 
accuracy of approximations to derivatives becomes particularly important because 
homoclinic orbit crossing can induce numerical chaos [12], Hence, it is desirable 
to have an approach that has global method accuracy and local method flexibility 
for treating the above mentioned highly demanding systems. Spectral element 
approaches [50], which combine the spectral method with the finite element 
method, have been studied for this purpose. A general difficulty in this approach 
is the interfacial matching between various spectral subdomains. This leads to a 
great reduction in accuracy. We find that DAFs provide a robust alternative 
approach for solving such problems. 

A second way to construct general DAFs is based on the variational 
principle also described in Ref [28]. A cost function X is defined as 
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2 



(69) 



where w is a positive weight function, {^j} is a set of functions and {aj} is a set of 
expansion coefficients to be determined by local minimization of X {{ocj}; x). It 
turns out that for an arbitrary local integrable function the set of coefficients {ttj} 
that minimizes the A. ({ttj}; x) is the same as that which provides an orthogonal 
basis expansion of the delta distribution under the same weight w. We refer the 
reader to Ref [28] for more details. It is noted that in this approach, each point 
can have its own basis set, independent of the basis sets used at^ neighboring 
points. This is equivalent to using a different basis for expanding at each grid 
point. Since this result is very general, we refer to it as the "DAF variational 
principle." This approach has been shown to be extremely powerful in molecular 
potential surface fitting [51], as well as in a variety of signal processing 
applications. Similar approaches exist in the mathematical literature; for example, 
moving least squares have been discussed by many authors [52], 

Orthogonal basis set expansions of the delta distribution have been 
described in very general form in Ref. [28]. A historical summary of orthogonal 
basis-derived DAFs can be found in the reference section of Ref. [28]. 
DAFs Generated by Nonorthogonal Systems 

1 . Fourier Space Regularization 

Definition 8. A tempered distribution on /? is a linear mapping T: S i? 
such that for some positive integer N and a constant C 

\T[<I>]\<C sup 1 x'^SVC^) i V^G5. (70) 

The space of all tempered distributions on R is denoted as S". The 
examples of tempered distributions are all polynomials and all L\ functions and 
even the periodic delta distribution ^^.-^^ ^ (Y^tnk) . However, 'neither 
^kg— ^ ^ fy-27T^t) nor e'"' are tempered. Since DAFs are functions in the 
Schwartz class, tempered distributions are particularly important to DAF 
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It is obvious that = 1 for all a. We construct the Shannon Gabor 

wavelet-DAF as 



SsbwD{^\<r,v)^T*w. ^^^^ 



This is a smoothed function, which has the property that 

Ssgwd{0 \ <r,r}) ^ 1 (75) 

The function ^zC^) is a regularizer and 

(76) 



as o - ~. We thus recover exactly the ideal low pass filter function %.C-h,v<') in 
the limit of ovu: regularization. Moreover, for a given a, 



^'y^ (77) 



as required for DAFs. 

The inverse Fourier transform gives the Shannon Gabor wavel'et-DAP [33] 

Wi?(x I = :F-MWd(^ I <>■,'?))(«) = 2^r«; = ^^^^e"^ (78) 



This is still a two-parameter DAP such that 

Urn 6sgwd{x I cr, j;)] = \/a,x e R (79) 
Jm Wo(x |cr,J7)] = 5(x) Vi7,x6i2 (30) 



The Shannon Gabor wavelet-DAF can be regarded as a Gaussian regularized 
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Dirichlet continuous delta sequence in the coordinate representation. 

In numerical applications we choose ^C^-^^^ "^y^-/} for a discrete 
version of Shannon Gabor wavelet-DAF 

r / I AN 1 sin [f (x - x„)] 

5sgwd{x - x„ 1 a, A) = - i^^,^^) (81) 

This form has the interpolating property. 

Example 2, Generalized de la Vallee Poussin DAF 

We choose as the following 

1 \i\<ri 

T„AO = < ■n<\i\<M . (82) 

0 otherwise 

where hlo^>>> Let tM.y^fll^W^ with ^ given by Eq. (73). Then we can 
construct a corresponding DAF by v V tu 

TT (A -1)77x2 ■ 

This DAF reduces to the Shannon Gabor wavelet-DAF, Eq. (78), in the limit of 
/^'^ J and to the de la Vallee Poussin DAF [32] when . ^ "'2^ . The 
latter has been tested for some numerical applications, including.solving PDEs 
[32]. 

Example 3. Arbitrary Schwartz class filters 

As discussed in Ref. [53], to design a general, smoothed filter, which can 
be low pass, high pass, band pass or band stop, we choose a Schwartz class 
function LU^ o which satisfies 



/ ^U)di = 1 



(84) 
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0 < w^{x) < 1 Vx 6 /Z. (85) 

A . 

Equation (84) implies that U^Co) :r 1 . Let Tg^.^- C^) be a piecewise 

smooth function characterized by a set of parameters \ and satisfying 

0 < T^,.a„...(0 < 1 Veeii. (86) 

Then we have a generalized Schwartz class filter 

Sdaff{x) = :F-Mr * ^l(x) = 27rT(x)to(x). (87) 

This obviously includes Example J and Example 2 as special cases. 

Another interesting special case is given by choosing 
.1 /3<UI<a a>0>O 



T..a(0 = 



(88) 
0 otherwise 



and given by Eq. (73). We then have a smoothed high pass filter 

Shpf{x I o-,a,^) — [siii(ax) - sin(j9x)]e"^^ (89) 



We note that this is equivalent to a DAF-wavelet. Equation (89) reduces to the 
Shannon Gabor wavelet-DAF when ^ - ^ . 

The method presented in this section is very general for the construction of 
arbitrary smooth filters and DAFs. (The desired degree of smoothness is obtained 
by appropriately choosing the regularizer.) We refer the reader to Ref "[53] for 
more details. 

2. Interpolating Formulae and DAFs 
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Since a Fourier space convolution leads to a coordinate space product for 
a certain class of functions, it is clear that regularization in Fourier space by a 
Schwartz class function is equivalent to a coordinate space regularization, by a 
Schwartz class function. Therefore, coordinate space iS-functiqn regularization of 
commonly used interpolating kernels leads to smoothed kernels, or DAFs. The 
most well-known such kernel is the Lagrange interpolating formulae 

in(/, x) = f: li{x)f{xi) Vx.. 6 (a, 6) (90) 

where Z( Of) is the well-known Lagrange kernel given by 

W= n Vx.-,x,-6(a,fc) (91) 

(Note that by using the summation convention, Eq. (90), all kernels and DAFs 
presented in this and the next subsections differ from those in the previous sections 
by a factor of Vl , where h is the grid spacing.) Lagrange interpolating 
formulae yield polynomial approximations to f(x) when its values are given on a 
set of n node points {y^}. If the nodes are chosen to be equidistant, 
y i'^''IA'iU-t)/h^l^ is 1^1^^^ A , then integration gives the famous Newton-Cotes 
quadrature formulae. We have constructed Lagrange DAFs by introducing a 
Gaussian weight to the Lagrange interpolating formulae 

The most important difference between our Lagrange DAF and the Lagrange 
interpolating formulae is the way in which nodal points ^ X J are chosen. The 
nodes of the polynomials are always chosen around each central point 

^ C^ih) ■ Therefore when Xc is close to the boundary a or sortie nodal 
points will fall outside of (a, b). This causes our Lagrange DAF 

approximation to have translational symmetry and be asymptotically unbiased in ■ 
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the whole computational domain (a, b). This way of choosing grid points is due 
to the fact that DAFs are essentially a (-g?o^<^ ) domain method rather than a 
bounded (a, b) domain method. This reflects the nature of the Schwartz space 
weight function. However, being a (-co^oo ) domain method, does not mean we 
have to use the nodes at ^<=<=> and it also does not mean that we cannot use 
it for finite domain {a, b) calculations. On the contrary, the Lagrange DAF is 
more flexible for arbitrary finite domain computations, because the iS-space weight 
function makes our Lagrange DAF kernel an effectively localized one. The 
Lagrange DAF matrix is effectively banded. In principle, on a portion (2M+ 1) 
of all the grid points which are closest to 1^1 must be used in constructing an 
approximation to the kernel 'Slp/ypty^'^c.' 1 2>') If the nodes are chosen to be 
equally spaced, then we only need to calculate Sih^pCy^-K^ \ 7> ) once for all 

More general interpolating DAFs can be constructed by using more general 
Lagrange interpolations. Let X; (x) L=^l 2 , K-^ be a set of functions 
satisfying 

C^j^'(==i) = (93) 
As a result, the generalized Lagrange formulae 

will have the interpolating property, so that 

Ln{f,^m) = f{Xm)y m = 1, 2, • • • ,71, 

providing that the y J are n distinct nodes. A very efficient way for obtaining 
JLi Ci^ ) of the Lagrange type is to choose a nodal function y^^f Y ) and a basis 
function V^'(x) defined in an open interval around every node, and which have' 
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at least the first derivative at every node, satisfying 

;i„(x.) = 0, 0 <| /x'„(«i) l< <» Vt=l,2,...,n 
i/.(x;) = 0, 0 <| l< oo Vi = l,2,.--,»i 
./.Wt^O Vi#m. (95) 

The generalized Lagrange kernels are then obtained by making 

This satisfies Eq. (93) and consequently Eq. (94). A corresponding generalized 
Lagrange DAF can be constructed as 

r /^n(g)^(gt) .X 

ogldaf - -77— r— T-7-tU,(x), (97) 

where u> ^' CK^ ^ S is a Schwartz class function such that 

0 < Wi{x) < 1 and ix;,(x,) = 1 \/x £ R. (98) 

Eq. (97) is a general expression for a large class of interpolating DAFs. 
Example L Lagrange interpolation 
The Lagrange kernel, Eq. (91) is obtained by choosing 

n 

/*n(a;) = - Sj) and 

= X — s,-, t = l,2,---,n, then 




" x-x; 



The corresponding DAF given in Eq. (92). 
Example 2. Cardinal interpolating 

Considering a set of equally spaced nodes f?((^'i/)jiA.i?6-.W and 
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/in(x) = sm(— i) and 
Vi{x) = x-iA, i = 0,±l,±2,---,±oo; 
then 

(-l)'sm(fs) . 

= ^rl !a/ » = o.±.±2,...,±oo. 



(100) 



This is a different form of the well-known sine interpolating formulae and a 
corresponding DAF can be constructed as 

SsGWD{x'Xi\<r)^^ ^e""^ i = 0,±A,±2A, ■ • • , ±M. (101) 

This is a different form of the Shannon-Gabor wavelet-DAF. As in the Lagrange 
DAF case, we only include 2M + 1 nodes in the calculation. Later in this 
subsection, we show that this is also a special case of a generalized Lagrange DAF 
(derived for evenly distributed nodes). 



Example 5. Trigonometric interpolation 
Let IKM ^ 1 and 



i/.(x) = siii(^(x"X,.)) a;. = l,2,.-.,n 



2 

n 

1 

then 

''•^"^-2<(...)sin[K.-..)] ^'''^ 

is an interpolating kernel. Obviously it has the general form of a LagFange 
interpolation. We construct a corresponding DAF as 
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Eq. (102) is simplified for the case of equally spaced nodes 
){ .-=-if^l^ J 6=<?,K2'-.?w to give 

... sin[a^.(x-l^)] . 

'••^^^=(2. + l)sin[|(.-a^)]' ^ = . -(10^) 

This is recognized as the Dirichlet kernel arising in Fourier analysis. The 
corresponding DAF can be written as 

sin ^{x — X,-) 

Sdgwpd{x - a:.- I cr) = ^ (^3^^ • (105) 

(27i + l)sm^i5;^ 

This is our Dirichlet-Gabor-wavelet-packet-D AF, which was introduced when the 
connection between DAFs and wavelets was first made [32]. As pointed out in 
Ref. [62], the Shannon-Gabor-wavelet-DAF can be regarded a special case of Eq. 
(105) because 

SsawDix - X, k) = lixn " -i^ 

"-~(2n+l)sinii^ 

sin f{x - X .) 



AH of the examples in this subsection have been tested by numerical applications. 
They all have similar levels of numerical accuracy for interpolating futictions and 
for numerically solving linear and nonlinear PDEs. It is noted that the prescription 
given in Eqs. (95) and (96) is very general and efficient for constructing 
interpolation kernels. One can easily write down as many more interpolation 
expressions as are desired. As a consequence, one can generate various 
generalized Lagrange DAFs by using Eq. (97) with iS-class weights satisfying Eq. 
(98). It is beyond the scope of this paper to give an exhaustive list all 
possibilities. (It is also true that there are infinitely many orthogonal series 
expansion DAFs as noted in Ref [28].) 

The connection between interpolation DAF and the "DAF variational 
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purposes. This idea led us to introduce Gaussian regularized Lagrange sampling in Ref. 
[57]. 

Example L Shannon's sampling 

Shannon's sampling theorem is one of the most important mathematical 
foundations for information theory and various engineering fields. It even influences 
statistical mechanics and serves as an important motivation for wavelet development. 
Shannon's sampling theorem addresses the issue of constructing or recovering a 
continuous function f(x) on the real line ;c e R firom an infinite, discrete set of known 
values {f(xj} 

/w=J;„/w^^g^. /^b;, (ion 

where Bx/x is the Paley- Wiener space of functions band limited to T|, i.e., their 
momentum representations are identically zero outside the band r\h. This theorem 
provides cormections between experimental measurement (which is discrete in nature) 
and theoretical predictions (which is continuous in nature). The reproducing kernel 

sin7/(g-y) ^ ~ sin7?(x-gn)sin7/(y-an) 

is related to Whittaker's cardinal series 

sin77(g-gn) ^ (-1)" sin7?x ^^g^ 

7j{x - X„) 7/X - TlTT - 

and to the Dirichlet continuous delta sequence, Eq (20), for appropriate choice of T]. It 
is also known for generating an orthonormal basis for the reproducing kernel Hilbert 
space Bx/x. By setting r\ = n, one obtains the well-known Shaimon's father wavelet, 



x) = — . (110) 

TTX 



with the Fourier transform 4>W = X / ) And the mother wavelet- \|j(x) = 

as discussed in Section IE. 
Shaimon's wavelets are not efficient from a computational point of view because - 
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of their slow decay as x becomes large. This is implied from the ideal lowpass property. 
Mathematical sampling theory emphasizes the fact that expression (107) is exact. 
However, in the real world, since one cannot actually use infinitely many sampling 
points, the "exactness" in Eq. (107) is not physically realizable. Moreover, there is a 
well-known paradox [58] regarding the notion of band-limited signals. The usual 
definition implies that a band-limited signal is an entire fiinction, whose Fourier 
transform has compact support. However, an entire fiinction cannot have compact 
support unless it is identically zero in the entire domain. Therefore, it cannot be both 
band-limited and time-limited, unless it is identically zero. This is in contrast to the fact 
that physically reaUzable states are well-behaved Schwartz-space fimcfions, which are 
effectively, but not exactly, both bandlimited and time-limited [59]. The.bandwidths in 
the frequency and time domains are actually related by the Heisenberg uncertainty 
principle. To construct a practically usefiil sampling formula which does not demand 
infinitely many sampling points while providing as high accuracy as desired for an 
application, we employ the regularization procedure discussed above and choose 

w^{x) = e"^ (111) 



to smooth out the Gibbs* oscillations and consequently reduce truncation errors. This 
leads us to define an interesting Shannon-Gabor father wavelet (SGFW) as 



$(^) = !!5^e-^. .(112) 



TTX 



Note that the Shannon-Gabor father wavelet is different from either the Shannon father 
wavelet 4> or the Gabor wavelet € C*^ ) ces(» ). It is interesting to examine 

two limiting cases of the Shannon-Gabor father wavelet: 

lim $(x) = 



TTX 



and 



<r-o 27r<r 
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Shannon's father wavelet is recovered in the first limit (provided W -400), The second 
limit, Eq. (114), indicates that the Shannon-Gabor father wavelet is a delta sequence. 
Using this fact, we can construct an interpolating Shannon-Gab.or distributed 
approximating fiinctional (DAF) [33] 

Sdaf{x^ XnW) = — ;;f r-e (115) 

The advantage of this expression over Shannon's reproducing kernel, Eq. (108), is that 
Eq. (115) is effectively banded. The best a values form numerical purposes are 
determined by the dilation factor ^A(grid spacing). For a given A, there-is a wide range 
of o's that deliver excellent numerical results. This Shannon-Gabor waveletrDAF has 
been tested on many numerical apphcations and is extremely accurate and robust for 
numerical solutions of linear and nonlinear partial differential equations. 

Example 2. Generalized Lagrange Sampling 

Perhaps the most general sampling theorem is due to Paley and Wiener [60]. For 
an L2 function / which is band-limited to j], its valuey(x) at an arbitrary point x can be 
exactly recovered from an infinite sent of (not necessarily uniform) discrete "sampling 
points" [xj] 



(Note that this implies a value of A^^, one then constructs the following- Lagrange-type 
interpolating series 



fi') = i:fMSk{z) {xeR), . ^ (117) 

where 



is a Lagrange-type sampling fimction. Here G{x) is an entire fimction given by 
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G(x)=(x-«.)n(i-i)(i-^). 



(119) 



and G' denotes the derivation of G. Equations (117) is called the Paley and Wiener 
sampling theorem in the mathematical literature and can be regarded as a generalization 
of the classical Lagrange interpolation formula to the real line (R) for functions of the 
exponential type. Unlike the classical Lagrange interpolation formula, Eq. (11 7) contains 
infinitely many terms, and we stress that it yields the exact f{x) for all real jc. Thus, the 
interesting point is that the information of a continuous function (containing a compact 
set of frequencies) on the real line (R) can be entirely embedded in'ah infinite, but 
discrete irregularly placed set of sampling points (grid points). Condition (1 16) is the 
best one can have [60]. There will be an aliasing error if the grid mesh is larger than is 
allowed by condition (1 16) or if the function f(:c) is not band-limited to r|. The major 
disadvantage of Eq. (1 1 7) is that it converges slowly. In practice, neither computational 
nor experimental data can ever be obtained at an infinite set of discrete sampling points. 
As noted above, from a mathematical point of view, a band-limited (i.e., compact support 
in Fourier space) function cannot have compact support in the coordinate representation 
unless it is identically zero. From a physical point of view, physical measurements 
cannot be conducted for an infinite duration, therefore physically realizable states are the 
Schwartz-class functions [59], which can be treated as effectively band-limited in both 
the momentum and coordinate representations. This suggests that one can truncate Eq. 
(117) and still obtain reasonable results. A simple way of achieving this is to introduce 
a weight function Wjt(jc) into the right hand side of Eq. (117). (This approach can be made 
rigorous by introducing the regularization in the momentum space, as discussed above.) 
This leads to the approximate equation 



oo 



(120) 



A particularly robust weight function on the real line R is the Gaussian. - 




(0 < <r < oo). 



(121) 
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Note that the approximate equation (120) becomes exact in the limit that approaches 
infinity. Moreover, as Oj^ tends to zero S*'(jc)w^(jc) behaves like a "semi-continuous" Dirac 
delta function, 

1 G{x) Gix) 

This is effectively a delta function because 

Therefore the kernel S\x)wj^(x) of equation (120) can be approximated as an integral over 
x^, then one also obtains exact results as o^—^Oi^, For a finite set of sampling points 
{xj^k=l which are distributed in the nearest neighbor region of point Xj^, we have the 
following LDAF expression [29] 

WF(x-xjt|M,cr,) = n • (124) 

The Lagrange sampling theorem is very general and it includes Shannon's sampling as 
a special case of Xj^ = kA = -x.,,. Then it is seen that Eq. (119) becomes 



. sm 

= A 



(127) 



Taking account of G'C'O =(-1*0. the equation 118 gives rise to 



sin f (g - Xfc) 



(129) 
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Since the derivation is independent of the regularizer co, it follows that our Shannon- 
Gabor-wavelet-DAF can be regarded as a special case of our Lagrange DAF. 

Example 3, Dirichlet (periodic) sampling 

Both Lagrange-sampling ans Shannon-sampling theorems hod for band limited 
functions on the entire real axis R. On might wonder what happens for a band limited 
periodic function. It is reasonable to expect significant reduction in the number of 
sampUng pints required because of the periodicity. This is indeed the case. Star [61] has 
proved that if a function f(x) satisfies the Dirichlet boxmdary condition, is periodic in T 
and band-limited to the highest (radial frequency 27iM/T, then it can be completely 
reconstructed from a finite (2M+1) set of discrete sampling (grid) points [61] 

^ sin ff (x - Xk)\ 

^^^^ = ^ ^^''' \omA ^V 

jb=-Af (2M + 1) sm [^(x - Xjfc)j 

where A = T/(2M+1) is the sampling interval (grid spacing) and the x^ = kA are the 
sampling points. Using our standard arguments about the contradiction between the 
band-limited and the physical world [62,33], and invoking the regularization procedure 
discussed in earlier sections, we then construct the following approximate sampling 
formula 

/(^) « E /(^O ..^^/. ■ r ' . e 131) 
jb=-w (2M + 1) sm [^(x - Xib)J 

where W, the computational bandwidth, is smaller thanM. The sampling (rid) pints {x^} 
are distributed around the point of interest, x. Obviously when o-*«>, we recover the 
exact sampling theorem Eq. (130). It is also interestin to examine the limit of M with a 
fixed A: 

(2M + 1) sin [f (x - z,)] ^("^ p/l F 

Wecall ^^V-yic.-) ^ the Shannon-Gabor Wavelet DAF [33] (SGWD), which 
is a special case of our previous Lagrange DAF [29,62]. Both the DGWD and the 
SGWD are generalizations of the (infinite grid) sinc-DVR [49,63], X (which is also the 
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semi-continuous Fourier DAF on a grid [25]). The DGWD reduces to the sinc-DVR in 
the simultaneous limits of M-4«> (with fixed A) and o-^«>, and the SGWD reduces to the 
sinc-DVR in the limit of o-^oo; 

sin[^(x-xO] sm[|(x-x,)] . 

--'-<»^^]^^ - 

sin[f(r-xt)] 

In this sense, our DGW and SOW can be regarded as a DAF-windowed sinc-DVR or a 
regularized sinc-DVR. Due to the Gabor window, the regularized sinc-DVR matrix is 
banded. This endows the regularized sinc-DVRs with great potential for applications in 
large scale computations. 

Stark [61] has demonstrated that the Dirichlet sampling theorem can actually be 
derived from Shannon's sampling theorem by appropriately imposing the periodic 
boundary condition. Therefore, all three sampling theorems are inter-related and both 
the Shannon sampling theorem and the Dirichlet sampling theorem are special cases of 
the Lagrange sampling theorem. 

Example 4. Sampling theorems in polar coordinates 

It is not obvious how to generate a sampling theorem in noncartesian coordinates. 
In polar coordinates, the Fourier-Bessel series plays the role of Fourier series expansion 
in cartesian coordinates. The notion of a band limited function is still important with 
band-limited referring to the Fourier-Bessel series expansion of the function which 
vanishes outside a circle of radius y. For an isotropic function f{x) banded limited to y, 
Camo [64] showed that 



oo 

/(»•) = E/(«.Jt)5a(r), and (134) 

'''''' =^^y;^^M^^J^ (-) 

where is the ith order cylinder Bessel function of the first kind m^ajy is tiie Mi 
zero of J.(-). This result was extended to a real function f{T,8) whose itb Hankel 
transform has compact support: 
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/(■■■*) =__E^E/(<'i».^j^)&(r)0„(«), and (136) 

This form is useful for problems with circular boundary conditions. By our 
regularization argument, w propose a corresponding polar coordinate DAF as 

SiMir - otik, e-Bnl Qik, cr) = Si{r - aik | eik)SMid -0„\<r) (138) 

7^.>i(w)(a?,-r^)' (2M + l)sin[^^)' * ^^^^^ 



This is obviously simply a product of two DAFs. It is noted that because of an irregular 
distribution of tt;^, Qj^ is not constant and its appropriate values need to be specified. 

DISTRIBUTED APPROXIMATING FUNCTIONAL WAVELETS 

Efficient and general procedures have been prescribed in previously for 
constructing "delta sequence generated wavelets" from the various delta sequences 
descried in previously. These methods are applicable for generating wavelets from the 
various DAFs described in the last section because DAFs are, at least form the point view 
of functional analysis, a special subclass of delta sequences. We shall call the resulting 
wavelets "DAF-wavelets", a terminology introduced in Ref [32], where various DAF- 
wavelets were created for the first time by taking the difference of two DAFs. It is clear 
that the various methods described in the last section enable one to create as many DAFs 
as desired. Moreover, the methods prescribed in previously will enable one to generate 
infinitely many DAF wavelets from each DAF. Therefore, we discuss only briefly a few 
typical DAF-wavelets in this section. However, the discussion in this section is not a 
simple repetition of Section III because DAFs are multiparameter generalizations of the 
usual delta sequences. Being a subclass (Schwartz-class) and a multiparameter 
generalization of delta sequences facilitates conversion of-the DAFs irito a wide variety 
of DAF-wavelets. 



DAF-Wavelets Generated by Differential Pairs 
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A general express for various DAF wavelets produced by our wavelet generators 



G" is 



(140) 



The DAF SqafC^I w,a,p,"-) can be any DAF discussed in Section V. The computation is 
straightforward for all DAFs providing that m is not too large. In the case of Hermite 
DAFs,Eq. (58),wehave 



(141) 



where m, M/2 = 0,1,2,-. Here, some simple properties of Hermite functions have been 

used to simplify the results. The Mexican hat wavelet, fSi^ii-^^ )& ^ , 

and the Mexican superhat wavelet /SSfJ^Cix? ""::<^ "^3/^ ^ are given by 
To ,(xlo) and Yo jCxIo) respectively. In general, the series of Gauss-delta-sequence- 
generated wavelets, Eq. (47), are given as a special case of Yo^(xlo), m =0,1,2,'". 

Since all of our nonorthogonal DAFs have the structure Tw, Eq. (140) can be 
written also as 

t=o - 'J' - i - 1)! ' 

(142) 

for nonorthogonal DAF wavelets. This form may be useful for the implementation of Eq, 
(140). 

The moments M* of DAF-wavelets generated using our wavelet generators G*" can 
be calculated as 
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- f k( 5'" 5"-' N 



A: + 1 < 7n - 

(-ir^;^'—W(.|a.^....Vx * + + ,H3) 

0 

+ 1 > m, A " m + 1 odd 
This expression is modified by a constant if the DAF-wavelets are normalized in L^(R). 

DAF Wavelets Generated by Difference Pairs 

A second class of DAF-wavelets is generated, in general, by 

^daf{x I a,^, . . . ; a\^', • - •) = W(a; | (x,P, - • •) - W(x | a',^', • ^ (144) 

where at least one comparable pair of parameters, say a, a*, are different from each other. 
For the Hermite DAF, we have 



I = l.:cp (l^) J^^ (^)" -^ir. (145) 



The Mexican hat wavelet is obviously a special case of this general expression 
specifically Yhdaf(^ 1 2,o; 0,o). " - 

It is straightforward to generate wavelets from the Hermite DAFs of the Fejer 
type, Eqs. (66) and (67). Some simple examples regarding our Dirichlet<jabor-wavelet- 
packet DAF and Shannon-Gabor wavelet DAF are given in Refs. [32] and [33] 
respectively. Since DAFs already have been found to be extremely powerful for a variety 
of numerical applications, we expect that DAF-wavelets will play an important role in 
all of those areas where wavelet techniques are applicable. This is currently under 
investigation. 



CONCLUSIONS 

The general connection between wavelets and delta sequences (the sequences of 
functions which converge to the delta distribution) has been spelled out in some detail-. . 
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Qualitatively, delta sequences are father wavelets (scale functions). If a delta sequence 
is an orthogonal system, it is found to span the wavelet subspace Vq in a multiresolution 
analysis. Various delta sequences arising in mathematical, physical arid engineering 
applications are reviewed. 

A set of wavelet generators is constructed for converting delta sequences into 
mother wavelets. These generators are connected with an infinite dimensional Lie 
algebra which has an extremely simple algebraic structure and includes the algebra of 
translation and dilation operations as an invariant subalgebra. The corresponding Lie 
group provides the basis for a mathematical description of wavelets, which is more 
general than the usual translation and dilation group. A new set of orthogonal wavelets 
is found in the case of Dirichlet*s continuous delta sequence. The well-known Mexican 
hat wavelet has been shown to be a special case of a variety of the Hermite wavelets, and 
has been derived by two distinct approaches. 

The general connection between wavelet bases and frames and conventional 
polynomial bases was briefly discussed. Essentially, the father wavelet 
corresponds to the lowest order polynomial and all higher order polynomials are related 
to the mother wavelets, provided that the polynomials are orthogonal with respect to 
some weight. 

Distributed approximating functionals (DAFs) were defined as mulitparameter 
delta sequences of the Dirichlet type, constructed using Schwartz-class functions. DAFs 
were classified as orthogonal and nonorthogonal. The former are constructed by 
orthogonal basis expansion of the delta distribution, and the latter are constructed by the 
method of regulaiization. Both orthogonal and nonorthogonal DAFs are frames. The 
construction of orthogonal DAFs is briefly reviewed and more details can be found in 
Ref. [28]. The construction on nonorthogonal DAFs is described in terms of Fourier 
space regularization. A general and efficient procedure for generating interpolating 
DAFs is presented. The connection between the DAF approach and mathematical 
sampling theory is discussed in detail. Various examples are given to illustrate our 
approaches. Clearly, there are infinitely many more DAFs . which can be__easily 
constructed using our approach. For example, the well known formula IIjC0s(2'^;c) can 
be used to generate a DAF. It is not possible in this paper to enumerate all the various 
possibilities. 
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A method of creating arbitrarily smoothed and arbitrarily shaped window 
functions is briefly discussed, based on regularization. Smooth low pass, high pass, band 
pass and band stop filters are constructed as special cases. The desired degree of 
smoothness is attainable by using an appropriate (S or C") regularizing function. 

A variety of DAF-wavelets (wavelets generated by using DAFs) is constructed 
by using either our wavelet generators or the difference method. The Hermite DAF is 
used to illustrate our approach because in that case, analytical forms can be obtained 
easily. The Mexican hat wavelet is identified as a special case of the Hermite-DAF 
wavelets. We expect that various DAF wavelets will play an important role in a wide 
variety of numerical applications. 

All references (articles and patents) referenced or cited in this disclosure are 
incorporated herein by reference. Although the invention has been disclosed with 
reference to its preferred embodiments, from reading this description those of skill in the 
art may appreciate changes and modification that may be made which do not depart from 
the scope and spirit of the invention as described above and claimed hereafter. 
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CLAIMS 

We Claim: 

1. A method for data padding and noise filtering implemented on a digital 
processing device, comprising the steps of: " . 

a. defining a total data set as the collection of all known and unknown data, 
where the unknown values may be interspersed among the known values, 
or concentrated in a region adjacent to the known values, or a 
combination of the two; 

b. placing the unknown values of the data to be obtained by a padding 
procedure so that the total data set contains only equally 'space data - the 
data has a constant sampling interval and the total data set represents a 
function; 

c. calculating an appropriate difference between the true data and an 
approximation to the data for all known and unknown data values; 

d. minimizing the difference with respect to the unknown data values by 
iteration or using the calculus of variations to obtain algebraic equations 
for the unknown values or solving the algebraic equation for the unknown 
data values. 

2. The method of claim 1, further comprising the step of: 

e. removing a noise firom the total data set using a non-interpolating, well- 
tempered distributed approximating functional (NIDAF)-low-band-pass 
filter to form an approximation to the total data set. 

3 . The method of claim 1 , wherein the data is a time-sequence of multi-dimensional 
data. 

4. The method of claim 2, wherein time-sequence is a 2-dimensional sampling of 
an image at a fixed time interval 

5 . The method of claim 1 , wherein the data is 1 -dimensional data. - 
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6. A method for data padding and noise filtering implemented on a digital 
processing device, comprising the steps of: 

a. defining a total data set as the collection of all known and. unknown data, 
where the unknown values may be interspersed among the known values, 
or concentrated in a region adjacent to the known values, or a 
combination of the two; 

b. placing the imknown values of the data to be obtained by a padding 
procedure so that the total data set contains only equdly space data — the 
data has a constant sampling interval and the total data set represents a 
function; 

c. calculating an appropriate difference between the true data of the total 
data set and a D AF approximation to the data for all known and unknown 
data values; and 

d. minimizing the difference with respect to the unknown data values by 
iteration or by using the calculus of variations to obtain algebraic 
equations for the unknown values or solving the algebraic equation for the 
unknown data values. 

7. The method of claim 6, further comprising: 

b. when the known input data set comprises a finite number of sequential 
data values having an evenly spaced distribution, adding a gap of a finite 
number of unknown data having the same spacing or sampling rate and 
some other functional dependence or tail function; 

c. calculating an appropriate difference between the true data (known and 
unknown, including gap points and tail function points) and a DAF 
approximation to the data values including values at gap points and the 
unknown coefficients in the tail function or sum of basis functions and 
choosing DAF parameters so that the largest predicted or estimated data 
values do not exceed the largest known input values - and the Jowest 
estimated or predicted data values are not less than the lowest known 
input values; and . 

d. minimizing this difference with respect to all unknown values and. 
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coefficients by iteration or by using the calculus of variations to obtain 
algebraic equations for the unknowns or solving the algebraic equations 
for the unknowns. 

8. The method of claim 6, fiirther comprising: 

b. when the known data set comprises a finite number of evenly spaced, 
sequential data points, adding a gap with a finite number of data point 
having the same spacing or sampling rate and adding a repetition of the 
finite number of evenly spaced, sequential data points to give a periodic 
extension of the known input data, where a sign associated with the 
periodic extension is the same or opposite of the input data and-where the 
same sign periodic extension yields a symmetric extension of known 
input data and the opposite sign periodic extension yields an anti- 
symmetric extension of known input data; 

c. calculating an appropriate difference between the true data (known and 
unknown, including gap points) and a DAF approximation to this data 
and choosing DAF parameters so the largest estimated or predicted data 
do not exceed the largest known input data and the lowest estimated or 
predicted data are not less than the lowest known input data; and 

d. minimizing this difference with respect to the unknown data values by 
iteration or using the calculus of variations to obtain algebraic equations 
for the unknown values or solving the algebraic equations for the 
imknown gap data values, 

9. The method of claim 2, further comprising: 

b. when the known input data set comprises a finite number of evenly 
spaced (or sampled), but sequential data points, adding interspersed 
unknown data points to construct an evenly spaced or sampled data set to 
form an augments evenly spaced sequential data set, adding a ga_p with 
unknown data having the same spacing or sampling rate and then 
repeating the augmented evenly spaced sequential data set after the gap 
to form a periodic, evenly spaced extension of the known data where the 
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extension have the same or opposite sign of the augmented data set where 
a same sign extension yields a symmetric periodic extended data set and 
an opposite sign yields an anti-symmetric periodic extended data set; 

c. calculating an appropriate difference between the true data (known and 
unknown, including gap and interspersed unknown data) and a DAF 
approximation to the total data set and choosing DAF parameters so the 
largest estimated or predicted data do not exceed the largest known input 
data, and so that the lowest estimated or predicted data are not less than 
the lowest known input data; 

d. minimizing this difference with respect to the unknown'data values and 
gap and interspersed unknown data by iteration or using the calculus of 
variations to obtain algebraic equations for the unknown values or solving 
the algebraic equations for the unknowns; 

10. A method for image processing implemented on a digital processing device, 
comprising the steps of: 

associating data of an image with a grip of points; 

associating a measure of disorder parameter with the data of the image; 

padding the data and periodically extending it as in claims 4 or 5, using a DAF 
approximation to the data with the DAF parameters chosen so that the largest estimated 
or predicted data values do not exceed the largest known input data and the lowest 
estimated or predicted data values are not less than the lowest know input data; 

filtering out the noise by using a well-tempered NIDAF-low-band-pass-filter to 
approximate the total data set and calculating the disorder parameter as a function of the 
DAF parameters M and olA, The change in the disorder parameter as a fimction of dA 
(for a given M) will be used as the indicator for the optimum well- tempered NID AF-lo w- 
band-pass-filter; 

if desired, one can also use the periodically extended and filtered total data set to 
carry out fiirther standard Fourier filtering. 

1 1 . The method of claim 10, further comprising: 

categorizing the images and patterns using the final disorder parameter; 
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images or patterns with the same disorder parameter will be assumed to be 

similar; 

a catalogue of such images or patterns, and their corresponding disorder 
parameters, will be used to construct a comparison and categorization of surface 
roughness, as well as other properties that generate images and patterns; 

similar catalogues of such images and/or patterns of stressed materials will be 
used to identify incipient structural flaws or weakness; 

similar catalogues of fractured or on-going fracturing materials will be used to 
identify microscopic fractures or structural flaws; 

similar catalogues of patterns associated with any kind of -defect will be 
constructed and used for predictive diagnostics. 

12. A method for data padding and noise filtering implemented on a digital 
processing device, comprising the steps of: 

a. defining a total data set as the collection of all known and unknown data, 
where the unknown values may be interspersed among the known values, 
or concentrated in a region adjacent to the known values, or a 
combination of the two; 

b. placing the unknown values of the data to be obtained by a padding 
procedure so that the total data set contains only equally space data — the 
data has a constant sampling interval and the total data set reppesents a 
function; 

c. calculating an appropriate difference between the true data of the total 
data set and a D AF approximation to the data for all known and unknown 
data values; 

d. solving the algebraic equation for the imknown data values. 

1 3 . The method of claim 1 , further comprising the step of: 

e. removing a noise from the total data set using a non-interpolating, well- 
tempered distributed approximating functional (NIDAF)-low-band-pass 
filter to form an approximation to the total data set. 
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Fig. 1 The Hennite DAF (a) in coordinate space, (b) in frequency space, (c) the first order derivative in 
coordinate space. The solid line is for o=3.54. A/=120 and the dashed line is for cf=236. M=30 
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Fig2 Original Lena image (240x240). 




Fig.3 (a) The degraded Lena image, PSNR=22.14dB, (b) Restored Lena image, PSNR=30.l4dB 




; / 
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Fig.4 (a) The degraded Lena image. PSNR=I8.76dB. (b) Restored Lena image, PSNR=28.19dB 
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PRintsrval OTinbival 

FigJ ECG Criterion Characteristics for Diagnosis 




(a) (b) (c)- 

Fig3 ECG filtering (a) Original ECG (b) Low pass filtering (c) DAF wavelet net filtering 




(a) (b) (c) 

Fig.4 EMG filtering (a) Original EMG (b) Low pass filtering (c) DAF wavelet net filtering 
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(c) (d) 
Fig. I. Lagrange DAF Wavelets 
(a) GLDAF (b) GLDAF Wavelet (c) Dual GLDAF (d) Dual 
GLDAF wavelet 
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(b) 
Fig.l. 

(a) Original Mammogram (b) Enhanced result 
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(b) 
Fig^ 

(a) Original Mammograin (b) Enhanced result 
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Fig. I Image restoration from 60% impuke noise, (a) Corrupted image, (b) Filtering by Sun and 
Nevou's median switch sciieme. (c) Our filtering, (d) Our modified filtering. 
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Fig. 2 Image restoration from 40% impulse noise, (a) Corrupted image, (b) Median filtering (3x3) 
(c) Median filtering (5x5). (d) Our filtering. 
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Fig. I HcrmiteDAF(M=8, 




(a) (b) 
Ftg.2. Mammogram Enhancement (a) Original mammogram (b) enhancement 
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Fig3. £CG Criterion Charactenstics for Diagnosis 
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(b) (c). 
Fig.4 ECG Filtering, (a) Original ECG (b) low-pass filtering (c) our filtering 
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(a) (b) 
Fig. 1. jrband-timited interpolating wavdcts 
(a) Sine function (b) Sinclet wavelet 
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Fig. 2. Interpolating Cardinal Spline (£)=5) 




(a) (b) 
Fig. 3. Interpolating wavclcu by auto-correlation shell (/>=*3) 
(a) Daubechies wavelet (b) Dubuc wavelet 




Fig. 4. Lifting scheme 




Fig. 6. 



Frequency Response of Equivalent Filters (I>=3) 
(a) Decomposition (b) Reconstruction 
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Fig. 7. Lagrange Wavelets with D=^ 
(a) Scaling (b) Wavelet (c) Dual scaling (d) Dual wavelet 




(a) (b) 
Fig. 8. Frequency Response of Equivalent Filters (D=9) 
(a) Decomposition (b) Reconstruction 
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(c) (d) 
Fig. 10. B-Splinc Lagrange DAF Wavelets (N«4. ;7=2) 
(a) Scaling (b) Wavelet (c) Dual scaling (d) Dual wavelet 




Fig. 1 1. Frequency Response of Equivalent Filters (A'=4, 77=2) 
(a) Decomposition (b) Reconstruction 
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Fig. 13. Nonlinear Masking Functionals 
(a) Donoho Hard Logic Nonlincarity (b) Softer Logic Nonlincarity 
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(a) 



(d) 




Fig. 14. 2D Lagrange wavelets for image processing 
(a) Scaling, (b) Vertical, (c) Horizontal and (d) Diagonal wavelets 



(b) 
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(c) 

Fig. 15. VGN image processing for Lena 
(a) Noisy Una (b) Median filtering result (c) our method 
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1*) . 0)) 
Fig.l. ;r band-limited interpolating wavelets 
(a) Sine function (b) Sinclet wavelet 




Fig.2. Interpolating Cardinal Spline (Z>=5) 





Pig- 4. Lifting scheme 
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Fig.6. Frequency Response of Equivalent Filters (£)=3) 
(a) Decomposition (b) Rcconstniction 



wo 99/46731 



47 / 



62 



PCT/US99/05426 




wo 99/46731 



48 / 62 



PCT/US99/05426 





(c) 



■• ^•^P''"^L28range DAF Wavelets (A'=4.„=2) 

(a) Scaling funct.on (b) Wavelet (c) Dual scaling function (d) Dual wavelet 
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(a) 



(b) 



Fig.9. Frequency Response ofEquivalcnt Filters (AM n=2) 
(a) Decomposition (b) Reconstruction 




Fig, 1 0. Mother wavelet comparison (A/=4, 77=2) 
Solid: B-spIinc Lagrange; dotted: Gaussian-Lagrange 
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Fig. 1 1. Gibbs overshoot of the Sine FIR 
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Fig. 14. Frequency Response of Equivalent Filters (//«5, ;7=:.3) 
(a) Decomposition (b) Reconstruction 




Fig, 15. Mother Wavelet Comparison (//=4, 7=2) 
Solid: B-splinc Sine; dotted: Gaussian Sine 





(a) (b) 
Fig. 1 6 Dubuc wavelets (Z)=3) 
(a) Inner scaling (b) Boundary scaling 
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(a) (b) 
Fig. 17 Generalized Sine wavelets (//==2, rj^l) 
(a) Inner scaling (b) Boundary scaling 




(a) (b) 
Fig, 1 8 Frequency response comparison of boundaiy filters 
(a) Halfband Lagrange wavelet (b) Sinc-DAF wavelet 




(a) (b) 

Fig. 19 Target extraction from color background 
(a) Original pilot view (b) DAF-wavelct restoration 
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W (c) 
Fig.2 1 . Enhancement of Database I 
(a) Original Manimogram (b) Linear enhancement (c) Non-linear enhancement 





(a) (b) 
Fig.22. Enhancement of Database 2 
(a) Original mammogram (b) Enhancement 
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Fig. I . Cube model of RGB color 
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Fig.3. Hexagon projection of color tube 
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(a) (b) 
Fig. 1. ;r band-limited interpolating wavelets 
(a) Sine function (b) Sindet wavelet 




Fig.2. Fourier Gibbs overshot of Sine FIR implementation 




Fig. 3, Sine CutofT Wavelets {M=9) 
(a) Scaling (b) Wavelet (c) Dual scaling (d) Dual wavelet 




(c) (d) 
Fig.4. B-Splinc Lagrange DAF Wavelets (A^=5, 1^3) 
(a) Scaling (b) Wavelet (c) Dual scaling- (d) Dual wavelet 
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(a) (b) 
Fig.5. Frequency Response of Equivalent Filters (A^5, /;=3) 
(a) Decomposition (b) Reconstruction 




Fig 6. Mother Wavelet Comparison (//=4. ;7=2) 
Solid: B-splinc Sine; dotted: Gaussian Sine 




"xs n n tr- 

(a) ^t,) 
Fie 7 Nonlinear Masking Functionals 
(a) Donoho Hardlogic Nonlinearity (b) Softer Logic NonJineanty 
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(b) 

Fig.8. Mammogram enhancement 
(a) Original mammogram (b) Multircsolution enhancement by 
DAF-wavelet 
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Fig.2. Alternative representation 



Blue 




Yellow 



Fig.3. Hexagon projection of color tube 
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(b) 

Fig.l traditional trivial window 
(a) 1x1 window (single pixel), (b) 3x3 window, (c) 5x5 window. 
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Fig.2 Arbitrary quincunx extension of symmetric neighboring basket 
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Fig.3 Quincunx basket selection for the filtering . V 





(a) (b) 
Fig.4. Image restoration from 40% impulse noise, (a) Corrupted image, (b) Our filtering. 




(a) (b) 

Fig.5. Wavelet noise removal 
(a) Gaussian-degraded Lena (b) DAP wavelet thresholding (c) DAP v/avelet + quincunx filtering. 
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